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. I. -INTRODUCTION v 

Baokground ' " . 

> The Vocational Eduoation AQt'of 1963 (PL 88-210) 

encouraged states to develop vooational programs and 
services for ^unemployed youths ^h^* socially disadvantaged, 
and the handlpapped (Barlo^^w, 1976; Williams, 1971)* How 
vtpll lodal education agencle^s (LEA^s) Implemented programs 
and s er V ig^ & to meet the vocational, training needs of these 
populations was to be fletermined by periodic state-level 
evalua t ions ajid evaluation findings r eporteifl .^to Congress by 
the NatlQ_.nal Advisory Council on Vocational Education in 
1968 (PL 88-210) . " ' ^ 

After reviewing findings of th# Advisory Council on 
Vocational Education's Report (Essex, 1968), which ' 
questioned whether vooational programs had served special' 
needs populations, Co^ress required states in the 1968 
Vocational Education Act Amendments to, use specific 
percentages of federal funds for handicapped an^d ^ 
disadvantaged programs and prescribed new vocational 
education program, evaluation^ requirements (PL 90-576), 



However, as ^rioted by ^he General /Accounting Of flee ' 
(197ittr, Lee (1971) arid Wentling ( 1 9S0), evaluation 
requirements mandated In the 1968 Araendit^ents were a source' 

= p£ .Qonfusion for many stages: ^ areas of st^te evaluation 
responsibility remained unclear* federal , evaluation ' 

^re^quirements remained vague; and a clear definition of 
evaluation and evaluation procedures was lacking in the 
legislation.'^ . .l-' ' . ^ 

In preparation for the ig7& Hearings on 
Reauthorization of ^the 1:9|8' \A;fflendments , the , U.^^ Office of 
Education contracted^ with the 01 yrapus/Research Corporation 

/to conduct a national assessment of vocational education 

;^ro^rams fdr the handicapped^ and dlsadvant.aged , 

Suiflmarizing the general state of Tfon|.toring and evaluation 

of Vocational education programs for the handicapped,, the- 

Olympus Research Corporation Report ( 1 974 )• stated : ' 

Actual enrollment figures were not available, and 
in most states there was little 'information on^ 
^ completers, dropouts, and , placements * Follow-up 
data w^re not available in any state.^ (p.2Q6) 

■ - ^ . 

he Olympus Re search Corporation « s Report ( 1 976) on 



the status of vocational education for disadvantaged 
populations also found actual enrollment figures were -i 

' s ^ " ....... % ' ^ . 

unavailable and observed program evaluation at state 
education agency (SEA) and LEA levels wis unsystematic and 
inadequate : 



Considering the informality of the planning 
process it should come as no surprise that the* -— ^ 
monitoring^ and evaluation of programs' for' the 4/ 
disadvantaged was equally as Informal at b^oth 'the 
state a,nd local levels, Cp, 161) - ^ 

• In response to this continued lack of informat&n 
about the impact vocational education %nd the confusion 
of many states over federal .evaluation requirement^, 
Cengress prescribed at least 28 specific requirements ^' 
related to federal, state, and local program^ eval^aation 
activity in the 1976 Vocational Education kdt Amendments 
(Wentling, 1980), Focusing on program evaluation by the 
states, the 1976 Vbcational Eduoatlon Act Amendments (PL 
94-482) identified systematio program evaluation as a 
responsibility of the SEA (Federal Rules and Regulations, 



October 3, 1977) 



The State Board shall, during t^e flve--year 
period of the State Plan, evalilate in ' 
quantitative terms ^the effectiveness of each 
formally organized program or^ project supported 
by Federal, state, and" local funds. These 
evaluations shall be in terms of^ 



) planning, and operational processesi - 
results of student achievement; 
c) results of student employment success; and. 



d) results. of additional services, as measured 
by a, b, and. c of this section that the state 
provides to social .populations * ^ (pp, 53842-3) 



Although some states had 'evaluation^ systems in 
operation that needed only minor ohanges to comply with the 
1976 mandates, most states !^d to develop entirely new- 
evaluation syste/ns capable of systjemaWQally evaluating all 
vocational programs (Smith & Holt, 1979,, 1980| Wentling 
Russo, 1978). Consequently most SEA*s were faced with the 
CQramon pr^lem of developing, Implementing and/or revising 
f their res$>ective evaluation systems with only a , limited 
knowledge base available from other states* 

statement ^f the Problem 

Verpontj^ like many other states, did not have a system 
for evaluating special needs programs and services as 
■ 'mandated in the 1975 Am^endments* In an effort to comply 
with fe^e ^,ev al ua tion manB ates t he Vermont Division of 
iVocational Education recently contracted with the 
^n.iver si ty *o f Vermont tp develop a statewide system for' the * 
evaluation of handicapped and disadvantaged (rt/D) programs. 

One of the first cintivities under taken by the 
University's project ^UiTf was to search for informatipn 

■ 

* ^ . ^^^^ 

about strategies used by other states to evaluate H/D 

programs and services. Perhaps, other states had* 

suGcessfully used a particular evalui^ion strategy &nd/or L, 

had encountered probl^s wl^iFi an evaluation strategy that 



.,10 *^ . 



G©ul<d have provided yaluable data for th© prbject staff to 
oonaider in designing a Vermont system However * an ERIC ^' 
search of literature and telephone contacts with several 
ind i^v idual s knowl edgab le about vocational program 
evaluation yield^ed- only limited iriformatlon about other ^ 
states evaluation strategies* 

Confronted with this lack of datap the University's 
projeet' staff was presented with th^ problem of how to, y/ 
" obtain information about t^he types of strategies usad by 
other states to evaluate H/D prggrams that could be ^useful ^ 
in designing a H/D proV^am evaluation for /the Vermont 
Division of Vocational Education* • * 

Purpose of the Study 



^ The overall purpose of the study was to iden^t^^y 
strategies used in sertc ted .states for evaluating the 
effectiveness of H/D vodational^^edTjdation programs ^nd 
servioes as^tfquired by the 1976 VoGftional Education Act 
Amendments* 

More specyicallyj the two central objectives of thp 
study were : \ 

,(1) To identify the type's of evaluation strategi&s 
- presently being- used by rural states in 



assessing' the planning and operational prooesses 
of vocational eduQation programs and serviaes •- ' 
for handioappe*d and .disadvantaged persons^ and, - 
2, .To^ ma'ke information gainetf from the study * ' 
^ available for , use la-designing^ a statewide ' 
, system tor evaluating harid ^cappei' ^nd 
.disadvantaged vocational education programs In' " 
0 Vermont , - ^^^^ 



Major Questions of the Study 
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Ilie 1976 Amendments clearly Amphasi zed the role of the 
individual states In the overall evaruation of programs. 
In order to gain a more complete understanding of how each 
state had j-esp^nded to the evaluation mandates * the study 
^^ught to answer two major questions: * 
(1) What type^ of evaluation strategies we>e being 
used by. rural states to assess the planning and 
operational processes of programs and services 
\ designed for hindicapped and disadvantaged 
\.studehts enrolled in vocational education 
programs? 

^ C2) Were handicapped and disadvantaged programs and 
* • ' services evaluated as a part of or separate 
from, rural states on-going systems for 



s * 



1^ \^ 



^\ ■ . , ■ 12 



/ , \ ^ "evaluating all vooational eHueafeion prpgrams ' 



within the period* of their five year plani?^ ^ \ 



Limji tat ions b^fflSfle Study 

•■ .- •= ^^^^^ ... - ^- : . \ ... .. . ^ * ■ . 

> Initi^rty, the Unlversit-^.V^,^^^ staff was 

Interested in seQuring information about the* types of 

Strategies used ^y SEA »:S to ev aluate - H/J) prpgrams ^and:. 

servloes in -terms of program . plan ning^^ M 

^processes ^ results of student actjAev em en t , and .results of 
student employment subceas as .Required by the 1976 
Amendments, However, an arialysis of Ve^^^f'^^^^^k^ity 
Assessment System CQAS) For Evaluating Voaational EduQation 
Programs revealed information within the QASoould be 
adapted for determining results of H/D" student achievement 
and results of student emplqy^eht suqcess (Frasier & " . 
Albright, Note 1)* Basejd'^ri this analysis ,the^ ^^V^ 



Division of Vooationai Eduqatton and the University* s 
project staff decid^ to emphasize program plannihg and 



.,pperationaJ- proce.^ses in the design of a H/D program 
evaluation system for Vermont* Therefore, to provide the 
project staff with Information for use during development 
and refineraent;of a Vermont ^i/D prpgram evaluation system, 
the study' was limited to identifying SEA evaluation 



stratifies for asses'slng p^rogram planniftg. an^ op#rational 
proaesses, ' ^ ' = ^ 

: / " - ■ ' = ■ \ ^. -V- -.^ • . ; ./ .. 

* As the projeot staf jf ' was^ aoncerne|i with development of 
a SEA-leV©lH/D^ program evaluation system^ only SEA ^ 
personnel were Interviewed by telephone. Due ^to budgetary 
oonstraints. and time lines imposed by an imtnediate need for» 
in^formatlon during deslg^n of a Vermont H/D program 
^evaluation system , contact with persqns know! edg|Bble about 
sample states evaluation practiqes was limited to a single 
telephone interview with eaoh^ SEA * s consul tant for H/D 
programs. No attempt was made to inolude' LEA personnel . 



. • DEFiNITldN OF TERMS . • ■ • ■ . 

HandiQapped - "A person who is mentally retarded , hard of- 
hearing I deaf, speech Impair ed , visually handicapped,, 
seriously emotionally disturbed, orthopedical ly impaired, 
or other Health Impaired, or a per son; with learning 
disabilities, and who by reason of this handicap requires 
special education and related servdces in oi:der to succeed 

drt a regular* vocational pnogram^ and/or raqulres a modified 

_ . ■ ' . « \ ^ = > . , .... 

.vocational education program" ( Federal ^Register , October 3? 

1977:53864)^.^. vl / ■ : ^ ; / ' ^ ' 



Disadvantaged ^ "Persons (other than hand icapped per sons) 

• € - • . . . . • 

who Have academic or economic disadvantages and require 
special siervices, assistance ,^ or programs in order to . , 
enable them to succeed in vocational ed^tfation programs" 
< Federal Register, October 3* 1977) * \ ^ ^ 



Academic Disadvantaged ...means that a person: lacks 
"reading and writing skillsi lacks matHematlcal skills; 
" or performs below grade level. (p* 53864) 

Economic Dl sad van t aged . * .means i family income is at 
Or below national" poverty level* participant or 
parentCs) or guardian of the participant is ' 
unemployedi partiqipant or parent of participant is 
s recipient of public assistance! or participant is 
Institutionalized or under State guardi'ahship" (p. 
&3864) 
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Evaluation - "Evaluation ts the proae^s of delineating , 
obtaining , and a^pplying' desGriptive and judgemental 
• information; concerning some; objects ^merit as revealed by 
tts goals design, implementation, and results; and for 
purposes of decision making and aOGOuntab ility.^"^^ 
(Stufflebeam, 1976 ) % . , , " 

Rural States ^ States that, according to the 19701 Census, 
have more than 50 percent of their population living 
neither in central cities nor in suburban areas ( Br yan , 
1981)*. ^ '\ , ^ \. \ : • r; ^ 

Plannlns and Operational Processes - Thase terms will \ < 
include questions developed to identify program planning 

^ and operational processes in: H/D 'evaluatiOin documents (as ^ 

'listed in Appendix G); AND , program |fla^ 

operational processes such f Si 

"^^^^/^^^^ ) Quality and . availability of 1 n.s true tip nal offerings; 

"^.fe) ^Guidance, counseling, and' placement .and follow-up 
\ :^^\ ■ services ; r ■ ' . ■ . . 

% (3) Capacity and condition of facilities and equipment| 
^;)ift)V Employer participation in cooperative programs 
\: ' ' vocational education ; ' 
^ ^^^^v5) , Teac ratios^ and, - . . . 

Cei Teacher qualifications (Federal Register, October 3, 
\ 1977: 53842) , ' - ' 
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Speoial NeWde 



(53 . 



p u i^^i o n s ^ ^ 



Speoial needs 
Of minorfcy groups ^, 



^ r -.-^ 

populationa't4are K 



Mem-bars 

HanalQapped- persons* 
pisadyan^ged pirsons I and 

Persons of ^ llmf ted lnglj.sh-speaking ability . 
(Federal Register/ October 3, 1977:53843) 
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A review of literature relevant _tG - SEA evaLu3^ion 
praotlQes in vocational educa^on w^^fc oondubted the 
purpose of examining how states ha^^^ 
^evaluation of H/D v'ooational Jed uca^ ton' \prog rams 
preSQribed by the 1976 Vopation^^ Educaiion Aot/Amendm:#.nts, ^ 
The literature searoh was impiemented in^^two stages: ' ; 
First, a search litirite^to /state H/D program^ evalUatidn- 
practieesj and ^ ^second an expanded searohL ti1iat.,^nvolved 

all state-level vocational education pr ogriion avml^uation 

r ^ "1- ^ / ■ ■ . * ^ : ■ . ' " ^ -^^ v^'v; ; : ^ ,.■ 

practices - 



Program Evaluation Practlce§_ ^^ 



An ERIC search of literature in August ig8r\ 
specifically related' to SEA evaluation of H/D vocational 
^education programs since the 1976 Amendments , and an 
expanded ERIC search o f 1 iter atur e ^in January 1982 reiated 
to SEA evaluation of vocational special - neieds programs , 
disclosed only one related article by Wentling^ and^-Russp. 
(1978), This scarcity of information about vocational 
special needs program evaluation was also noted in a recent 
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^riyifw of SEA speoial needs^iprogram "©valuati^^^^ 

ifhtfelps (:ff80)\and. in a ritc^tfnt review of literiture on ehe 

I :"v--:;'A-^.V_^;- ;• ■ • _ ^; . ^ ; - 

; eyary^tion^^df focatlonai aduoatlon programs atid sapvices \ ■ 

■ -vt'^;.v' ' M'--^ •■ ^ ■ ,-. •■ - . /u ^ - '^-'-^ 

^ for^Llffilted^English-apeaking populations 'by Thoma^tand 
\:.Margoei,(:198?)^ Smi th ( 1 982 ) identified a^ eirailar lack of 

writing in \^the professional literature on tha\ evaluation 
: , practfpes used Iby . SEA * s to qonduet edUoation V 

^pr^T^v eval ^ ' . \ ^ 

General SEA Program ^Evaluation Pragtlqes 

v_ V A general survey of literature related to SEA ^ 
ev^al,uationf of vocational education programs ttriqe^ the ^ 1 976 

^ ADdendmenta identified a serdes of three studies, oontr acted 

■■■■ ^-'^ ■' ■ ' ' . • . ■ - _ ' . .. ^ - :.V- ■ 

by the Natipnal Jnetitute For Education to deterraihe the 

^.jirjag^^a-^o£^0atatis^.4:n im evaluation. f 

requirements: X3Lf_.th^^^ tAe ^Contnact Research 

Corporation national Survey on the s4atus of Evaluation ; 
activity across the states C Smith & vHolt, ig79| 1980)| the 
Abt Associates study of the extent to which the states had 
implemented various provisions of the 1976 Amendments 
(Beuke, tukas, Brlgham, Oliok & Brien, 1980); and, the 
Wentllng national survey of State DirectoV^s F9|r Vocational 

. Education to determine the extent to which program 
©valuation requirements had been , implemented in their . 

'13 ' " ^ - \ - - ' 
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respeotive itates CWentllngy igST | WenMlrig &4Barn 
198^) • . How^er^ o:£ thmMf wthn^ studies) Abtl; ati*- 

Wentllng stuiliea provided Informatiori atk>ut €he eKtant to 
v^htch SEA'S had evaluated |i^e results of addfltional ^ . 

services .'for speolal needtsi^opulations, . V = 

' Highlights of the Abt and Wentiing Studies \ _ ^ ' : \ _ 

The Abt Study - ^ The major emphasis of the Abt Study 
(Beuke et al.y 1980) was to. determinp the extent to wfiloh 
15 sample states had ImisleiAented provisions of tri^ 197^ 
Amendments r^^latfd to^r program improveftenti #duaa^^^^ 
equity I and the distribution of federal funds,\^ States were 
seleoted to ensur e : geographioal repr e^entation based on 
the four census regions ; perGent-of the total potential 
V o 0 a 1 1 o n a 1 _st^d en t _ pg pMl a tX^R I .i. a n# . - type ^of^-vOQatlonal 
governance structure used ^bV the ^^t^te to- adml 
1976 Amendments;, Data . fror^ sta-^e- pnd ^ loeal * levels were 
colleoted during the 1 979--8d school year by combining yease 
study and surv.ey methodologies. Interview data were 
supplemented by a'n analysis of . documents colleeted in" each 
of the 15 states, surveyed* ' - 

: Findings of the Abt Study (Beuke et al . , igfiO) related 
to the evaluation of . special nejeds programs and services 
disclosed i . ^ 



: S ;Ver|r little attention had -^been giv^ to evaluating 
the results of addltioriil programs and\ services ' ^= 
/ for special needs popul^lons • ' ' ^ 
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§; All^ sample states whiqh ifw^'a r^rmal pr 

review process inoluded some items on special: 
need.a^ioputationsr but' the extent of att^ 
given by states to" the eyaluat^ of sp'e^lal needs 
jpORUlaWons- programs ^nd ''services varied > 
considerably, \ ./ v ^ 

The pnlmajpy emphasis of SEA jevaluatipns related to 

- ■ .t^;r^a?^^ ■ -. ■ ■' \- . ' ^^-^'^^ ■ 

: special'^heeds. populations had . been on access to 

vocational programs and services , 

• ' i ^ — ■ ■ . = . ■ ^ - ' . . • ; ■ ' 

© Evaluation of spwial: populations »- a6to 

employment suooess had not been- conducted by 5EA»s 

in the 15- .states ^surveyed. 



There appeared to be, among those staters that gave 
m more; expensive evaluation, a widely used^ : 
evaluation strategy format of ^sequenced self-study, 
by the liocal school distric t , external review by a 
team of individuals, and development of a local 



15 



21 



/. -improvement pi an 'based ' on the results of the 

, - extspnal team neyf'aw. — • ■ ■ ■ a^. ■ 

• ■; r ' ■,■ ■f.f^^- , -J'^ ■• ■ 

' Ttfe Wentling Study ^ The major purf'^fe of the Wentli'ftg 

Study (Wentling, 1981; Wentling & Barnard^.1982) was to 

-•■ ^ - : ^ ^' - - : . ■ 

determine the opinion of State Dlreotors For Vooatlonal 

■ ■ " '1 . f V ' ^ " ■ ■ . ^ 
Education conqerning: 1) the extent to which the evaluation 

requlrements,*of the 1976 Amendments had been implemented in 

their respectlye states and territories | and 2) how they^ 

perceived the effectiveness and utility of vocational 

" : • ' -^Km^^-- ■ - - - . . ' ^ . ' " ■ - -^^ . ■ 

program evaluatioh. A, survey questionnaire, mailed to the 
state directprs With a. cover ' letter sigtied by the Executive 
DirectQr of the National Assooiatlon of State Directors of 
Vocational Education , was the data collection method used , 
in this study. The response rate was 88%, 

A review Of findings in- the WentJ^ing Study (Wentling, - 
19811 Wentling^ Barnard, 1982) related to eval^atibn of 
special ^eeds populations programs and services revealed i 
•By the spring of 1981, the evaluation of programs 
and s^ervices for speoiar populations was not yet 
fully implemented' in 2 9 states. ^ 

8 Very little had been done to evaluate the results^^ 
of programs and services to speolal populations. 
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A total of; 36 state dirfeGtors indioate^the^ 
continue to evaluate progranas and aervices for 
speGlal pop'ulations even If federal evaluatlbn^ 
requirements 'were eliminated , ' 



Hcrwever, a major limitation of the Abt and Went liiig 
studies was the aggregation . of data under the generic 
category of "speQlal needs populations**. Neither study 
delineated state evaluation activities' for- each special 
need population group. Consequently j specific Information 
related to how states evaluated handicapped and 
disadvantaged programs was lacking in both studies. 
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MEtHODS AN^D PROGEDlMs .' * " 

The analyaes in the/Sfcudy were based\on handloapped 
and d±sadvaftta||d program eu^alfuatlon matarlali reGe^ea \ 
from sample states and from telephone Intenviews condueted 
with H/D Qonsu&arits . follbwing a review of each state's 



evaluatioh documertts. Methods and pr oeedur es for seleoting 
sample states, eolleotlng and analyzing SEA evaluation 
documents^ and Gonductlng telephone Interviews are 
described in this section * 

Sample V ' 



^ *^ineteen rural states ( as identified by Bryan ^ 1981) 
formed .the target population of the study ,^ A stratified 
single-stag€^ sampling method adapted from Moser & Kal-ton 
(1972) ^?as used to assure an even geographic spread of 
selected states, 

target states wert alphabetically ordered , numbered 1 
through 1 9^ andi divided into four groups according to the 
four census regions I Northeast, North Central, South and 
West (U.S* Bureau of the Census). The regional grouping of 
these target states is shown in Appendix A. 
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^ , As the Northaast Region hiad only three states and ' 

^" " ' ^ ^ ' - - • . " ■ . y ^ , \ ■ •■ 

Vermont w^g excluded from the selection -proaeBS, two states 
were selected from tills group. A table of random numbers' 

^ . v.- ^ r ■ / -v - : f^ : 

Was used to Identify four ;states in eaoh ffroup until a 
tgtal of 14 states were selected as the sample* ; 



n 



Colleotlon *of WD Evaluation documents - ' 

A telephone re^quest to individuals in;3EA*s 
responsible for H/D vocational education programs (H/D 
consultants) was the primary method used to obtain 
evaluation documents,* A personal letter (shown In 
Appendix B), was sent to each state H/D consujtint ^within 
two days following the first telephone oontactb 'The- 
letter, adapted from Erdos ( 1 970) , reiterated ^the purposes 
^f the /study,, identified the H/D documents requested j and^ 
expressed appr eoiat Ion for the Ind Iv iduall s antio ipat ed 
Qooperatlon, 



*Note^ Two U,S, Office of Education Direcljbrles wepe used 
to Identify H/D consul tants of sample staties • (1) The 1981 
Directory, of State Of fleials with Responsibilities for 
Vocational Education of Disadvantaged Youth| and, (2) The 
1981 Direotory of State Personnel Responsible for 
Vooatlonal Education of Handiea-pped Persons, 



^If state ievaluatlon documents were n<f€ receiv|!€^wl thin 

j^:^ ■ ■ ■ ^ . . . " ■ : ■ V, ■ • ' ^ ; ' 

15 working, days aft,er*the Initial tielephone request", a 
follow*-up .telephone Gont.a£fc wag made to ihquipe whet*her the 
requested doeumerits had: beem sent* . A' third and f ihal 
telephone inquiry was tnade when the do^urtiejits -were not 
received Wi^thin 30 wprking days al'ter the initial telephone 
'*re quest, " . ^ *.* , ^.'^ ' ^ ■ 



Design^ of Instruments .. . ' . w 

■ / ^ ^ ^ . / - ' • • \ ' 

• ■ Two categories " o'rf content analysis , the evaluation of " 

program pi anning 'and the 'evaluation of program* operational 

processes , were used in the study . Conceptually , these 

categories direc-tly corresponded to two types of evaluation 

within the CIPP Evaluation Model (Stuf f lebeam , Foley, 

Gephart, Guba~ Hammond, Merriinan & Provus , 1 971 ).i 

evaluation of program' planning prac esses with - Context 

Evaluation; and evaluation o'f program operational pVoce^ses 

with P - Process Eval-u'^ion . ' ^ 



Con text Ev aluat lo n provides informatibn for 
planning ^deeisions by describing actual cond itions , 
determining unmet needs and unused opportunities,'^ 
and provides general information for determination 
of program goals, (p, •218). ' \ 



Process Evaluation provides informatio'n foV 
impleraenting features of the program, potential 
sources of program failure, and provides general 
information for determination of program 
improvement. (p. 229) ^ 
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Questions for iifen:tifying the evaluation of H/D ^ : 
program planning and operational processes were formulated 
by g two-step process adapted from Stuf flebeam * s (1976) • 
framework for designing evaluation studiesT""^^^^ 

Step 1 i. General questions were delineated' for 
evaluation of program planning ( Context-Evaluation) 
and program operational pro^eesses (Process ^ 
^ Evaluation), . ; 



Step 2, Specific questions for identifying the 
evaluation of H/D program planning processes and 
program operational processes were formulated from 
the general questions developed in Step #1, 



^ ' A similar two-step process was also suggested by 
Wentling (1980) fof use in developing questions to evaluate 
the effectiveness of occupational education and training 
programs , ' - 

^ the format of "IF YE3-then"lqu#stions, was adapted from 

Stuf flebeam» s et al , (1971) cohtingency mode of context 
evaluation* These questions , written to reflect the 
meaning of each program planning and operational process 
q^ufestion within the abnteht analysis instrumentSj were 
formulated to probe for types of i \ a) evaluation designs. 
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used to implenient H/D program evaluations j b) strategies - 
used to "verify how" a program « a planning and operational 
prooesees wer© being evaluated; and, e) strategies used to 
' "verify the effebtivenesa'^ of a program » s planning and' 
operational processes, ' - ; • ; ^ 

.Possible^ strategy types commonly usfd In evaluation 
were list0 under each " IF YES-then" : q faoilitate 
ease of record ing information and to maintain continuity / 
within each category of analysis. The formating of 
strategy types. under each^^ 

Validation of Instruments 

Internal and external review procedures adapted from 
Sanders and Cunningham (1973) were used in the development 
o^ the content analydis Instruments to maximize th 
potential of questions for identifying H/D program planning 
and operational prooesses/ 

Two Internal reviews of the content analysis 
instruments were conducted, by selected faoulty members at 
the University of Vermont. The first internal review was 
done by three Individuals vSo were selected for their prior 
'work with hand icapped and disadvantaged programs and/or ■ 
experience with- rural educational programs. The second 

internal review was Qarrled out by another two^ individuals 

' ■ . . - ■' ' ' 

■ . ' ■ ; : ■ . 22 , , ■ ■ '■■ 
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seleGted for their expertise in evaluation of eduoational, 
programsV , . 

An external review was conduct^ by an independent 
panel of three^xperti. Experts were selected who had ; / 
previously adapted the CIPP Model Vs (Stufflebeam et al . , 
1971) conQeptual framework for designing vpoatlonal ' 
edUQatlon evaluation systems , and/or who had sppciallzed in 
designing evaluation systems for vooational ^ special needs 
programs , 

Throughout the internal and external r^iew process, 
each person was given the same set of instructions for 
reviewing the content analysis instruments, IndividiJMs 
were requested to critically review and comment on the 
appropriateness of I a) each content analysis question for 
identifying H/D program 'planning mM operational processes 
and to offer additidrial questions\that could be used to 
identify such processes I and / b) the "IF YES - then" * 
question format for identifyirig ^types of SEA evaluation 
strategies, A sequential record -of internal and external 
reviewer suggestions and the action taken in response to 
emdh reviewer's comments is presented in Appendix D. / 

The content analysis instruments used in the study are 
shown in AppendiA E (planning jSrocesses) aiTd in Appendix F 
(operational processes), 



states H/D evaluatloft dQcuments ware analyzed using a 

" ^ - . ^ ' \ ' - / • ■ ■■- 

method of quaX Itative analy.sis for determining the presence 
of particular contetit ( Berelgon i 1 971 ) . . The prootedures 
employed are described in ssquenee belowi - 



. 1 



Before starting anal^ysis of a state's 

. - ' ■ ' ' . ^ ' ^> ""■ _- ■ ■ " . .. 

documents, the evaluatipn materials were read by 

the in vest igatp r in order to gain ^ an apprec iation . 

for the organization, content and Implementation 

procedures of. the staters evaluation system, j 

2* Each atate* s documents were ;first analyzed for 
evaluative questions (hereafter referred to as \ 
indicators) indicating SEA efforts to evaluate ^^^^ 
program planning proGesses and , then, for SEA 
^ efforts to evaluate program operational processes* 

... ; 



3, To simplify use of the Planning and Operational 

Content Analysis Instruments, the presence/absence 
of each bategory * s ind ieators (as suggested by = 
Wentling, Note 2) were recorded on Code Sheets aS: 
"shpwn in Appendix G, 



4. Af ter Gomplstion of the cbntent analysesy the , 
^ marked Qode sheet indiGators triggef ed ^the^ - 
' • ^^leetioh of app Analysis 

Instrument IndiQator Sheetp (Appendix H) within 



1 - . 

the content analysis Instr^u^ . 



5* Evaluation strategy types were identified for eaeh 
^ category: of atialysis^ indioator and recorded on the 
■ ap propriate Content Anjtl y sis ^^I^^^ ,,:3h f e t s^/ ^ _ 



When the 'content , analysis of . all states were 
Qorapleted^ a twowstep analysis of data was conducted. 
First : State Category of Analysis Indicator Sheets were 
sorted by nuftiber, ' Second r The types of evaluation 
s^fcrategles used by each state, as marked under "IF YES ^ 
then" /questions^ were compiled in descriptive form . 

Establishing iriter^rater Reliability ~ / 

: ; .Inter-rater reliability was estaiblished through 
adaptation of a method suggested by Thomas (Note 3) for 
determining agreement between individuals using content 
analysis instruments. 



' Before a sample staters dp [ ■ 

non-involved: state* s H/D evaluation docuraents weT# jbintly 
analyzed Jby an independent rater, and^bhe Investigator*^^. - 
^Throughout this, joint analysis , each rater helped to train 
the, other byi openly sharing Informationi discussing use of 
the instruments ; clarifying the Intent of questlons| 
diseussing and resolylng differences | and 'noting reasons " 
for similar judgements. ^ . \ ^ 

; Following the. Joint anaiygis, the Investigator > worked 
alone to conduct a state-by-state dontent analysis of H/D 



evaluation doouraents. After the Iny estigator had completed 
the analysis of documents from all sample states ^ the 
||independent rater randomly selected and anaiyzed two . 
state* s evaluation documents ^ number of agreements 

betweeTi raters divided by the number of agreements plus the 
number of disagreements were^ oalculated to give the 
perceTitage score of inter-ratter rellabil Ity for each 

content analysis instrument. - . \ 

■ ■■ - ■ ' ■ ' . - 

\ " J / ^ ■ ■ ' _ 

Telephone Interview s 

■ ■■ _ - » . - - 

U/D consultants of sample states were interviewed by 

phone after the content analysis of documents had b^en 

completed, Indiyiduals interviewed were thbse who had been 

previously contacted for information about their respective 

^ - 26 
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■state's evaluation of- H/D 'vocational programs/ Before. eteh ' 
■ Intervi&w beg'&ni the purposes of the study were reiterated, . 

, questions related to the study ahswered / and v^ei^bal ^ . ' 
assurance was giv^en to the person ihterviewad that ' 
individual and state names w^^'d ^ hot be identified in 
reporting of the study* ^ ^ 

The telephone interviews with individuals ""whose atates 
did not have formally structured H/D /program evaluation 
instruments or procedures were ^ery liraited (App^endix I), 
The que stions asked werejmerely Intended to : (1 ) verify, the 
state' s lack of a formally structured ^H/D program 
evaluation system;* (2) inquire whether H/D programs wfere 
evaluated as part of^ or separate from., the state* s 
on-going ayatem^ for evaluating all yo'catlonal education 
programs within the period of the five year plani and, (3) 
^ ask about plans ;the state might have had regarding the 
evaluation of H/D programs and ^services* < 

A more extenfslve telephone interview instrument was 
deyeloped for s t a t e a . w i t h f o r m a 1 1 y structured H/D program 
evaluation inatruments and procedures. The first part of 
the interview was intended to : / 1 ) verify how the documents 
received were currently being used (e,g, under revision, no 
longer in use); (2) inquire whether H/D programs were 
evaluated as part of, or separate from, the state's 
on-going system for evaluating all vocational 'education 
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programs within the period af the five year ^lanj and, C3) 

. 1 ■ ^ ^ " ■ ' - ^ ' ' ^- - ' \ i ■ ' ' ' -. . '■ - 

if nepessary, oiartfy H^ms about' the state' s H/D 

- ' ' ' ' '. ^ - . '' - * • 

evaiuatibn system that had resulted from^ the content ■ 

analysis of SEA evaluation documents^ The second part ^ 

. . ^ ^ ■ ^ ^ y \ . ■ ' ■ ■ ^ ■- ^ ^ . 

Involved a set of questloiiSH:< Appendix J) that wa.s Intended 

to probq for the opinion of H/D consultants Gbna©rnlng: a) 

- - - ' ^ '■ " ' ■ 

purposes for evaluating special' need s^ programs | b) utility 

of their respeotlve state » s evaluattop instruments and 

procedures for determining the effectiveness of H/D progran 

..." . - * 

planning and operation al pr ooesaes | o) :^^use'" of evaluat ion. 



resul ts by the SEA and local education pgencles | and d) 
difficulties encountared In developing . and In ImiJlementing 
their respective state' s H/D evaluation instruments and 

procedures, ^ . 

ti ^ . . ■ ' . - . 

Telephone interview d ata were^ tabullted in descriptive 

form^ state by state, and used to supplement each state's 

document analysis. 
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^V. RESULTS 



The telephdne requealto sample st^^tes for H/D progrm 
evaluation documents re^Ullid U receipt of materials 

from 13 of I4^etates. ' 

Four categories of program* evaluation aGtlvity were 

atatesi (1) states vrith agparate, formally structured , H/D 
program ^ evaluation Ins^rUnients and proc^sdurea; C2) states 
^wlth fGrrmally structured H/D program evaluation instrum#nt3 
and prooedures integrated Into their tota^al vooatiortal 
education program evaluation system; (3S> states condugting. 
H/D program evaluation p bait v^lthout forEaally str udtured 
Inatruraents or procedures; and , (4) stafces not evaluating 
•*^H/D vocattonal education programs* How^^ver, a fifth 
- category of SEA pr ogram • %Viluation, aotiv-^ ity wa a added as a 
result of telephone interviews with the state H/D 
consultants. This fifth catigory oonsi&^ted of states in 
the procesa of developing i H/D program evaluation systeni , 
These five categories of SEA evaluation activity are listed 
In Table 1 , 
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liable ' " 1 . 



R^Q^llEvalUate^lon Activity' 



CategOTy 



er of States 



States with separatf 5 f^M^i^^^^ 
H/D^progpam eTOluatiort ff^itm 



States Mth fonmllged H/Bprosc^an 
evaluation Inte^ated l^ifcothe 5 
ye^ state evaluation sy^ti 



States conduotlng prog 
evaluation but without^ ^ 
f ornall^ad systCTi 



States in the process of davelopln&s 
a E/D pro^*am; emluafciori Bptm 



States not conducting H/to pTOprwi 
evaluation 



Total sftates 
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To; faoilitate clarity in the repor-tlng of data . 
analsrsis findings, :the four states with separate, forniaily 
structured, program evaluation Instruments^^ 

procedures and; the one state vJlth H/D program evaluatloh ^ 
. instruments and procedures "integrated'' into the total SEA 
vocational program evaluation system were combiried Into a ' 
single: category entitled , "SEA'S with formally structured 
H/D program evaluation systems. In order to maintaim the 
anonymity of these five states and sustain continuity in 
_J!efigrMng_j:j^tm^ 
assigned a number from 1 to 5 . 

Findings reported In this section have been ordered to 
reflect: (1) the results of document analyses for states 
with formally structured H/C program evaluation system^V 
(2) th# results* of H/D consultant telephone Interviews for. 
states with formally structured H/D program evaluation 
systemsi and, (3) the results of H/D consultant telephone 
Interview s for st ates c onduc tin g- evaluation, bu t without 
formally structured H/D program evaluation Instruments or 
procedures. 



; 1, Results of trie Content Analysesy*^ 



^ 'A. Prbgram Planning and Operational ProQegaes. 

A Qontent analysis of materials nltaeivad from five 
sample states with; formally strUG^' H/D program 

evaluation systems revealed each state's doeuments had 
evaluative questions for assessing program planning and 
jQpmj:B%iQjasiL^r:(^.9MBBem^^^^ by _ j 

inier-rlpter reliability scores of ^ percent^, using the 
Content Analysis Instruments for Identifying Program 
Planning Processes, fc^nd of percent', using the Content 
Analysis Instruments for Identifying Prdgram Operational 
Processes* / V 

The presence of states . evaluative questions for 
assessing program planning and Operational processes are 
identified by category of analysis IndiGators In Tabie'^2. 
In terms of program plartning processes , .the three most t 
ooramonly used Indicators identified i 

© whether the unmet needs of an individual student 
influenced the student's pr^ram goals arid 
objectives I . 

V 

© Whether the Individual s knowledgable about the nee 
* ■ . - - ■. . 

of H/D students were Involved in planning that 
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influenced formulation of the LEA program's goals 
.and objectlvesi and , • ^ , 
I .whether the local H/D ^rograin was serving only those 
■ students who were unable- to suooeed in their regular 
^ voo at ional , education : pM^og ram , 



In termB of program operational processes, the two most ' 
conimonly used indicators Identified t ' ' 

e whether, people invoivec^ in .and/or affected by a H/D 



.1 

instructional resouroes available for use by the 
participants; ai%d,^^ 
i whether a program 'instr—uctorC s) had the neoessary 
^ teacher qualifications to work with handicapped 
and/or disadvantaged students* 



B, Program B¥aluatlon D e signs - ' 

Through the^ content anaiy— sis af H/D evaluation ^ 
'dOQurnents, two groups of H/D p rogram evaluation designs 
were Identified : centralised and decentr al ized , St^ates 
with centralized Evaluation di-aigns had direct , on-site ^ 
SEA involv.ement during evaluation activities and the final 
py-pgram evaluation .report was'^^r itten by the SEA. 
Conver^selyj states with deo6n t&"arized evaluation designs 
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minitaized direct SEA inVolvement during evaluatioh 
activities and the SEA did not write the final program 
^valuation report ' / ' ' 

The evaluation deiigns of states #1 and #3 were 
'Qeritralized . In both states, the program teacher was 
responsible for conducting a self-^valuatiori using^ SEA 
prescribed instruments Findings from the self-evaluation 
were then verified by the H/D consul tant during an on-site 
verifioition team visit held to confirm all vocational 
teacher self-evaluations. The final program^ evaluation 
report was written, by the H/D consultant using findings of 
the on-site verification visit.. 

States #2 and #4^ had^ecentralized evaluation designs. 
In each pf these states, a local evaluation team (its 
compQsition based on^ SEA. guidelines) was responsible for 
conducting an evaluation of the program using SEA 
prescribed_^struments . Foliowing the evaluation, a member 
of th^loQal team wrote the final program evaluation report 
' and submitted it to. the H/D consultant, / ^ . ' 

The evaluation design of state #5 was centralized , but 
a mixture of SEA and LEA d Ireoted program evaluation 
activities. In this statei an inter-tdisc iplinary , 
self-study evaluatlbn comm^it-tee seiected by the LEA (its 
composition based on SSi guidel lrils)-j>^^^ responsible for 
conducting a school-wide, evaluation- of all supportlv^^^^ 
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services provided. to vocational special need? students. . 

After completion of this study, an SEA. appointed external 
evaluation team, using the saine type of SEA evaluation , 
instrument as used by the LEa', conducted an evaluation, of 
. the school's supportive services' to special needs students. 
However, the final evaluation report was written by'^ the 
state •oohsultant for* vodational special needs using' (a) 
the results Of the LEA self^studyi <b) the voaatlonal 
^special needs ex ternal evaluation team's findings; and (g) 
the. reports of two other exter^nal evaluation teams that had 
assessed regular vocational 'programs and the school '^s^ 
cpmplianoe with state and federal vocational education 
laws . . . / ^ 

. ■■ ■ ■ ■ . ;■■ ■ = ^iZ^^: - ;. ^ ■ . - - ■ • ■ 

The analysis of - state* s H/D program evaluation designs 
revealed #ach of the five sample states with formally 
structured H/D program evaluation systems: 

§ utilized SEA designated Likert-type scales to 

quantitatively record the extent of program planning 
and operational processes I 
® used the same evaluation design fo^^sessing the 
effectiveness of program' planning ^^^^esses as it ' 
used for assessing the effectiveness of program 
operational processes' and, 
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• i amployed.the same ©valuation design to evaluate H/D 
prograihs ^^it used to evaluate other voeational 
, edUGation programs . 

C. Program Evaluation Strategies . ' 

Evaluation strategies in four of five states involved 
the use of an external IndividualC s) In the evaluative 
process. In the on ate not invplvlng an external 
evaluatorC s) , the evaluativ^ strategy was to survey 
indiylduals involved in and/or affected by all vdcatlonal 
prbgrams^i ^ 

The four external evaluation strategies identified to 
assess the effectiveness of H/D program planning and 
operational processes were: 

1* The H/D Qonsultant ( state #1 ) . The consultant 

aonduoted an on-site^ visit to view t^he program in 
operation, talk with the program's students and 

I Instructor, and review program and stud^ent records. 
Bas^d on this visit,, the consultant verified the 
program inatruQtor' s self-evaluation of prog^m 

1^ ef fee tlveness . ^ 
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A apeQlal needs "expart" appointed by the >H/D 
GOnsultant C^tate.#3), The expert was appointed 
from a group of Individuals seleeted and trained bv 
tht H/p Qonsultant. The Gbnsultant or expert 
coridUGted an on-site visit to view the program in 
operation, talk with the instruotpr, and review 
program and student reoords*^ Based on this visit, 
the evaluator verified the- progriim instruGtor's 
sel^f-evaluation of the program»s-effeQtlveness* 

TAn LEA appQlnted teani of individual sV Its " - - 
ooropositlon preSGCibed by the SEA, employed in 
voGations not directly .related to edUGation (state 
#4). Usually, this team has been Gomposed of local 
,H/D .program advisory aomAlttee members* An evening/ 
meeting of team 'members was held to disouss SEA 
formulated guestions about the program and to rate 
the ^prpgram * s ef feG tiveneS'S^ ' ^ 



An SEA appointed team of speGial needs experts from 
outside the LEA (state #5). Experts from other ^ ' 
school distr iGts - persons employed as yoGatlori'fel - 
speoial needs InstruGtorsor in ocGUpations related 

the edUGatlon.of.^ d isadvantai^^ 

individuals - GoridUGted an o.n-site visit, ^ 



interviewed gtudants and the program staff , viewed 

. ■ # - ... 

the program in operation , and reviewed program and 
student reoords. ^ Based on this visit ^ the team - 
members evaluated the program's effaQtiveneas using 
a SEA developed instrument. 



^.v; In the one state nd^t^tiivolvlng an external, evaluator 
(state #2) J the LEA oonduoted an opinion survey (its 
composition determined by the SEA) of individuals involved 
in and/or jaffeeted by vocational eduoatioji programs. Four 
groups of individuals ( randoffily selected by SEA prescribed 
percentages) were surveyed: " parents, stu^nts, school 
.professional staff and advisory committee members. 
Instruments were individualized for eaeh survey group and 
pontained questions j:elated to the effeotiveness of all 
vocational eduoation programs* individual responses were 
tabulated j recorded as gr^up mean scored and reported on an 
^^EAxvpneser ibed^ form J (^^^^\ ^ i - 




2* ^Telephone Interview Results For States With Formally 
' Structu red Ev i^luition Systems . 

, - ^ " the H/D consultant telephone interviews for states 
-wi-tir_f o rm alXy^v^ tr ue t ur ed _ HZa . pro g ram .„ev al ua t i o n . g y s t em s 
revealed a generality positive opinion about the purposes, 

' 39 ^ : ' 



fffeotiveness, uWllty, and value of their' respective ■ 
state's H/D program evaluafcdon system-. 

When asked whether H/D pro&raras were evaluated as part 
of, or separate fromi their State* s on-going system for 
evaluating all vooational education- programs within the 
period of their SEA/s Five .Year Plan, consultants in four 
stated (states #1, #3, #4, and #5) answered they evaluated 
H/D programs as a separate activity and with separate 
instruments. . The^ consultant, of state #2 responded H/D 
program evaluation was included within - was part of - the 
SEA* s evaluation in s^ument use evaluating ali 

voaational eduoation programs. All oonsulfcants IndiGated 
H/D program evaluation was oondijated at the same tinie as 
the SEA was evaluating all other voaational education 
programs as required in the 1 976 Amendments . 

Ih response to the^ question : "What are 'the purposes, 
for evaluating H/D programs and services in your state?", 
all consultants stated it was to provide information for 
program impro^ment. Thirteen additiorial; purposes related 
to program planning and ope^ra^onal processes were also 
identified by consultants. . These purposes* are listed in 
Table 3. ■ , 

On a scale of 1 to with 5 being the highest, 
consultants— assigned a mambe^^^^^ 3 or 4 when asked 

how well their state*s H/D evaluation system Was. able to 
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H/D Gonsultents^^liilons About the 
^o^pQBmB for Evaluartlrig M/D Trog^mm 



Consultant optalons by rta 



Sfl S#2 S#3 S#4 



To detennlne vftieth^ the 

is saving H/^ students X 

To deterniine whether the la doing 

what It said It would do to meet the 

neMi of E/D stiMentB Ce*g, meeting 

objectives stated in the fUrrilng pTOposal X 

To provide infoniatlon for pro^^ 

iiqprovOTent X X 

To eraluate p^ogPOT pl^mto^ prooesees X 
To identify student process — x 

To identi:^ teaching nanagCTient pa^oblons X 
To identify iis^^loe needs of H/D 

instructcrs X 

To min Informtlon of what. Is actmlly 
happening in H/D prolans ^ for tofonnlng 
the state leglBlatiff'e and ^rent ^oups X 

To be accoimtable fca* use of feds^al fi^s X 

Td determine whether the prop^m Is 
^ovldlng quality level servloes to 
students ^ . 

To determine whether services are being 
provided to B/D stijaents in a 
non-tKr^tenlng environment 

To meat federal evaLimtlon requJr^ents^ 

T6 help the"I£^'^et^'mine lo^ ---- .-^-^^^^^^-^.^.^l^..: 

pro^^i needs 

To deter^itoe what the pro^^ is doing 



Note* S - state 
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determine the ef f ©dtiveness of looaL program planning^ 

H - 1 - " ■ 

prooeases. When aaked to rate their system's ability to 
determine the ef feativeness of iQeal program operatioafel 
prooesses y consultants of states with an -on-site^ S^^y 
appointed^ evaluation team oomponent assigneci a nuraerioal 
value of 4»5 and 5f while consultants of • states without ^:an 
on-site^ SEA appointed , evaluation team oomponent rated 
their system* s ef feQtlveness as 1 and 2.5* Consultant 
ratings are gr^aphlcally illustrated Lipi Tab'le 4, 

When asked whether their state had encountered any 
diffioulti%s during developraeht of its H/D pH^Pram 
evaluation system I all Qonsultants responcfed they were 
unaware of any particular problems or were not employed by 
the SEA at ^he time the system was developed.^ However i two 
consultants mentioned they had* encountered dif J'ioul ties 
during the implementation of their staters H/D program 
evaluation system,/ The epnsultant of state #1 disclosed 
the staters H/D program evaluation system did not readily 
apply tOj all H/D programs and that more than one system was 
needed for evaluation of H/D .programs and services. The 
consultant of state #2 outlined three difficulties 
encountered by the SEA in using its survey instruments: (1) 
the LEA'S found it time consuming to administerj collect 
and tabulate the questionnaires; (2) the LEA* s heeded more 



Consmtftnt .Mttogs of How Wall. TO® Ij* Syetan (to " 
Assess Jt*ograin PlanMng ar^ jC^sratteml K^oceises 



Typm of K^oceSB 



Consultant rattagi by state 
p#l vS#2 : S#3 S#J4 . S#g 



Ftogmn plannljig processes 4 . 4 3 4 4 

]^g^am operational processes ^ 4*5 , 2*5 5 1 5 



Note * S ^ Stat« 

Ratings are on a soale of 1 to 5 ^th ^ 
5 betog the highest* '. ^ 



help from the consultant to interpret survey results than ^ 
SEA time and travel funds could providej and, ^(3) the 
questions In the surveys were- too complex or too technlQal 
for some -respondents • ; : 

With the exQeption ofVpne s^tate»s difficulty in using 
Its 3urvey^j3.nstru|^^J;^^ a belief 

that reactftd^> -p^ their state's 

evaluation of H/D programs had been good to ^xoellent. 
When asked the question : "How do you as a state consultant 
think the local level vocational educators use results of 
jH/D program ev aXuKti^ns? " , - tlii following f^^ ~ 
received : ' 



"They take it to heart and 90% of the LEA»s 
implement recommendations. We know this because our 
State Advisory Council for Vocational EdUQatlon did 
an evaluation of how.LEA^s perceived our H/D 
evaluation.system." (state /n) 



"They take th,e evaluation seriously , It has 
resulted in people from the Gommunity becoming 
Involved in planning proaesses ana using program 
advisory committee members, ^ It gives a real purpose 
to advisory committee meetingsV" C st^ate #2) 



"Yes and No, If they like the evaluation , they seem 
to^ keep the program. If not, they drop it and make 
a new application for another handicapped or 
disadvantaged program.. They don*t feem to like 
improving on what they have." (state #3) 
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^'They attempt to implement reGommendatlons beoause 
it is their own local people making reaoramendations , 
not the state." (state #4) 

"The LEA'S implement them. If they don't we don't 
'fund the.prog^ram again." (state #5) 

• ■ . .■ ■■ ■ ' , " r > ' . . \ 

„ In response to the questioni "How do you as a state 



. oonsultant use results of H/D program e^^aluations? " ^^11 
Gonsultants reported using results for monitoring whether^ 
program evaluation reaoramendations had^ beefr^mplemented by 
the LEA, Consultants of states with aentraliged evaluation 
designs indicated using, program evaluation findings to make 
.recommendations for program improvement and , when visiting 
H/D programs^ to determine whether . these^recorafflendations 
had been implemented. Similarly^ the consultants of states 
with decentralized evaluation designs, when visiting H/D 
programs / mentioned using results for determining whether 
the LEA had Iraplemented recommendations made by the locally 
appointed program evaluation team. The specific responses 
of consul tan^ts are ilsted in Table 5. 



mhlm 5 

feportad Usa of 
ftHDp^ Eroluatlon ResultB 



Results fepOTtad 
by CcaisultantB 



To nBka recCTniCTidations for 

pPO| 



3#1 



mz ms s#4 



Tb sea If reoonmendatlons far 
K pTogmm ^mrovmmmt have bean 
g( Inglamented whan visiting the 
program 



To see if looal reeonmendatlans 
mada by tiha loorily appointed 
evaluation team have been 
InrplCTiented whan visiting the 



So daval<^ a state plan of action 
for worlcLng with the LEA 



To da tOTntaa whether the progt'a:n 
vd.ll be refuMad > 



To-daslgi^toserviQa -pro^OTs-for 
E/D Inst^uetors / 



Notap ^ S ^ State 



S * Te 1 e pho n e In t e r v 1 e w Re su 1 1 s Fo r St a t e s Co nd uo 1 1 

fivaluatione But WlthQut Formally Struoturad EvaluatlQn 
Instrumenta or FroGedureg ^ ' • . . . - — ""^'^ — ~ 

The H/D Gonsultant telephone *lritervlews for states 
□onduQtlng H/D program evaluations , but without fdrmairy - 
,^ structured evaluation instruments or proGedures, diseloaed 
eaeh of these states evaluated H/p programs: Cl) at the * 
same trime as all other vooatAonal education programs | and^ 
O)^ with instruments and/or probtdures different than - those^ 
used in the state Vs on-going system for evaluating all 
vooational ^eduoation programs * ^ . 

During telephone interviews , consultante in these 
states revealed they controlled the evaluation design by 
acting as the external evaluator. The consultant: a) made 
an on-site visit; b) determined whether the program was c 
meetings Its goals and objectives as stated in its funding J/ 
propo^salj and, cX wrgte the final program evaluat,ion 
report, - , 

Consultants In this group expressed they knew their 
H/D, programs well and / thusV^d^ not need to conduct an 
evaluation using formally structured Instruments and/or 
procedures. Some consultants referred to' using H/D program 
standards "if necessary" to aid in the .evaluation of a 

.''BBTMi^lS^l^^PJ^^MX^^-^^^.^^ they eyaluat^jed. 

programs through informal visits and observations of the 



program over an indefinite period of time, 'THffe 
consultants expressed a personal desire to hav-e a formally 
struotured ^H/D program evaluation^ system, but each was ^ 
doubtf ul ' whether federal and/or state funds would be made 
available to develop and implement suoh a system*^ 

Telephone interviews identified three states in the 
prooes^ of developing their H/D program evaluation systems . 
The consultants of two states dlSGlosed their SEA*s were in 
the process of developing entirely new SEA evaluation 
systems based on results, of Federal monitoring reviews by 
the U*S/ Department* of Education* The other state's \ 
consu,ltant disclosed a new SEA evaluation- system was being 
developed to meet evaluation guidelines recently mandated , 
by the State Director of Vocational -^Education. : 

"One st&te didr not evaluate H/D vocational education ' 
programs. This stat^lSi consul taht s the .^A had never 

conducted evaluations of H/D programs in the past ^ nor was 
the oohs:u^tant aware of plans to conduct such evaluations 



.V. CONCLUSIONS 



■ : - The evaruation Of lla«d^capped and 
vooational education a^Cared to be an organized aGtlvity 
in most rural statea the sample 

states,: 10 of 14 states , ey^uated H/D prbgrams . 

i /Although all states , evaluated H/D programs at the same 
3 . as . a 1 1 ^ the r- y oc a tlonal ed uc a t io^-fl^pg ram s we r e t eing 
evaluated /only o^ne^ state evaluated H/D programs as part of 
its totals SEA . voGational education program evalua^tion 
system, " in^thfi^other nine states, H/0 programs were 
evaluated separately, apart f^om, other voQational ^ ^ 
eduoatioh prbgrams . / t 

' The predominate evaluation strategy used for assejssing 
H/D programs was tb have an ex ternal evaluator ( s) visit ^ 
eaeh' program to talk with the instruotorCs) , interview 
students,' view program facilities and records, observe the 
program *in operationi, and report findings to the SEA. In 
the one state without an e^cternal evaluaton component, the 
strategy was to InGlude H/D program assessment questions 
within thie SEA'S opinion survey administered • to individuals 

involved in and/or afffioted by all vocational eduoation 

■ ■ ^ -' ' ■ - ■ '- ' — — - = — =— = — ' 

-programsT 
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^ased on the analysis of doouraents provided by states 

with formally structured evaluation instruments and 

procedures, the .types of strategies used for atfsessirig H/D 

.program planning and operatlohal prooesses were determined 

by each state's preferenGe for either a SEA controlled 

(centralized) or a LEA Gontrolled (decentralized) 

evaluation design. States with a centralized evaluation 

deaign utilized oontrol strategies for conducting 

evaluations which overtly Involved the H/D consultant in 

all phases oT each program evaluations, limited the extant 

of LEA involvement in program evaluation activities, and 

placed responsibility for writing the final program 

evaluation report on the H/D consultant* In contrast', 

states with a decentr al ized evaluation design utilized 

control strategies conducting evaluations which 

minimized involvement of the H/D consultant in program 

evaluation aotivltiesi encouraged LEA involvement during 

program evaluation activities, and excluded the H/D 

oonsultant from participation in preparatioh of the final 

program evaluation report* 

However, regardless of whether. the SEA preferred a 

centralized or a decentr allzed evaluation design, all SEA'S 

with formally structured evaluation instruments and 

procedures used the same strategj^.^ -Pnr nnntr^lllng the- — — 
. ' ■ ^ ' ^ ^ ..... ... 

content of evaluative questions and the recording of ' 



evaluatop's assessments. In each statei this control was 
effeGted through the joint strategies of requiring an . 
evaiuator.CsX to use SEA prescribed i (1) questions for 
assessing the effeotlvenessbf program 

operational processes p and, Likert-type scales for 

recording their opiniona about the t¥6^nH|^pf H/D program 
planning and operational processeS^, 

All states inducting ^ H/D program evaluation but 
without formally structured instruments or pp^ocedur es had 
centralized evaluation designs* toong these states , the 
sole evaluation strategy was to have the H/D consultant act ^ 
as an eKternal evaluatori but without formal SEA strategies 
for assessing the effectiveness of program planning and 
operational prOGesses. In essence , the H/D consultant 
determined the conduot, content and recording of program 
^evaluations on a program-by-program basis* 
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VI. ^DISCUSSION AND RECOMMENDATIONS 



The number of operatlohal procasa IndlGatorB reGorded 
by the researoher appeared to have little relationship with 
Gonsultant ratlngB of how effeGtively their evaluation 

" - • ■ ^- ^ ^ ^ . .... . 

systems were able to . assess . -H/D program operational 

- ' ■ . ^ ~ " " -V- 

proGesses* For Ejcample i State # 3 had six Indloators and 

^ state # 5 had ^ne indicator of program "operationai '^'J-- 

proG^sses (see Table 2 on page 33)* But, ori a soala of T 

to 5, 5 being the highest ^ each of these state *^S: \ .7 

consultants rated their evaluation system's e f fee jtio^en^ss^:^^ 

as: «5" (sle Table 4 on page 43)* Apparently, regardlrii of / ^ 

how many indicators were present i in the SEA * s' ^yaruatl^^^ 

'doGUments, sbme other factorC s) must have Inf luenoed the ' 

ratings of consultants about the effectiveness of their V :,., 

^-ystems for assesslng^ program operational ^processes.: 

One influencing factor appeared to be the extent of 

SEA Involvement in aotivlties during assessment of LEA ^ ^ 

program operational processes* . In states with an on-sitei 

SEA appointed, evaluation team compohent ( stat;^s: # 1 , #3 and 



#5), consiJltants rated their evaluation system as^ very 



effective for assessing program . operational processes^ 
But, In states without an on-site, SEA appodn^d. 
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evaluation; team component C states #2 and #4 ) / consul tants ■, 
rated their system as less; than ^effective for asaasslhg j.; j. 
prdgram operational- processes . 



This observatlpn suggests the ri;^ed for further 
research to det ermine whether, the LEJljjs, of samplers tales 
would agnee wi th, r;ep6rted consultant ratings of their SEA 
evil uatlon system' s effectlvenesg* IfV fotr example, LEA's 
of states mth centralized evaluation; designs Cstates #f, 
1?2 ana #3 5 repisrted direct SEA invol^emente^lnor^sed the 
.^^ffep«^neS8 of ;prog evaluations ?^d the -LEA»s of ; 
states with decentrallz.ed evaluation jdesigrfs (states #2 and 
m) identified the lack of direct Se£ involSvenient decreased 
the effectiyenesst of program evaluations, then the degree 



of SEA tnvolvemeht may *proportionatei|^ increase program 

evaluation effectiveness. Perhaps, if SEA ^nvolvement ^ 

-^^P*'?^^^^ the effectiveness of progra^ evaluations, then 

' ■■■S^^^ '5 X evaluation, rttivlties should be 

• ■ . ■ ^ -".if . ' 

^part of each state»i program evaluatlSn design . 
. ... \ . / . ' . \ ^ - 'i: . r 

~: ^ Another discr^f pahcy ^in the studyjdata became a^pparent 

when''consultant opinions about the '•piirposesV for 

conducting ^evaluations were compared with their opinions 

about how they as consultants "used" program Jevaluation 

■"results. of . the 14 purposes for evaltiating H/D: programs 

V reportedr by •oonsul tants (see Table 3 pji page '11), only the 

purpose of program ei^aluation "to provide infordation for 
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prograin improvement" : was mentlpned by all consultants. 

However,- as dan be seen in Table 5 on page 46, Goneultants 

us^d program evaluation results for making program 

-improveraent r eooramendations and for .determining whether 

prograrn improveraent reGOmmendations had been implemented by 

the LEA'S. Evidently , although not mentioned by 

consultants, another purpose of program evaluation was to 

.provide criteria for de^rmining whether LEA «s had 

implemented program improvement recommendations . ^ 

Based on this analysis of data, it was unclear to the 

researcher whether consultants were providing technical 

assistance when they visited programs ( e .g , offering 

equipment modificatipn suggestions and cur riculum revisions 

to meet the learning needs of a handicapped student), 

and/or ,^ whether consul tants were making compliance checks 

when they visited programs Ce,gp monitoring the extent of 

program improvement in order to determine if the program 

should be refunded) . Perhaps, as reflected in the - 

following comments'" of consultants abouft how they thought 

LEA*s used program evaluation results, SEA'a mostly "used" 

results to provide information for compliance checks^ 

"LEA'S implement them ( recommendations for program 
improvement)* If theyyon't, we don't refund the ^ 
program again" (p. 44)| and, 

"If the' LEA likes the evaluation, they seem to keep 
the program. If not^ J^h^y d^roj^i^^^^^ 



^ application for another H/D program. They don Vt 

s&era to like improving on what they had", (p.45) 

However, if SEA* i use -program evaluation resulti for 
the predominate purpose of^roviding information to make 
program compliance checks, LEA's may simply be making 

program impr.ovements in order to secure refunding or 

- j/i^' ■ - 

terminating programs to avpid SEA monitoring practices. A 
study focusing on LEA program implementation practioes 
could be Helpful in defining the state' a role in working 

with LEA *s about program improvement matters. . ^ ^ . ... 

To developers of state s^ystems for evaluating H/D' 
progr;ams and services, the following^ general 
recommendations are offered t ^ 

§ Identify the SEA's specific purposes for evaluating 
H/D programs and services* ^ 

® Determine the extent to which the SEA^ will be 

directly involved in evaluation activities Ce*g. , 
^ during LEA orientation meetihgs, during the actual 
conduct of the evaluationi during preparation of the 
final program evaluatlon^report) # 

Establish with the SKA who will be responsible for 
the actual conduct of the program evaluations (e.g. 
the H/D consultant, a LEA or SEA appointed 

- 55 



evaluation team, an" Indepandent third pdrty 
"evaluator) . 

i Have the SEA define and enumerate how prpgram 

evaluation resulti will be used after programs have 
been evaluated. 

9 Build into the atate^E B/D program evaluation aystei 

a meQhanism for assessing the Irapaot of program 
V ^^aluation on th^^^^^ ; 
services for handloapped and disadvantaged 
vocational edljcation students. 
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POUR/G^RAraiC DIVISION OF Kffi UOTim STA^^ 



Am RtffiAL STMES l^j^^CH DIVISION* 



M0 



tfortheait 

Maira ^ - ^ ; 
New te^shlre 

' lowa^ 

^ &uth DaJcota 

Soifeh 

Arkansas ' 
Mississippi ■■ 
■ Kentuol^ • . 
I^rth C^^ollna 
South Carolim 
West yirgij:^a 

weBt\ \ -y . - ' 



Percent of , 
Population' 
OutsMe Cent^yL 
Cities am ^ ^ 
, Si^i^s ^ 



79 
' 73 
100 



65 
58 
57 
88 

m 



Alaska 
Itoio 
Montana ^ 
New Mejdloo 
.Wyoming 



m 

69 
82 
60 
62 
61 
68 



100 

. 84 

75 
69 
100 



te .fo'ur geog 
County aod C 3 



pMo divisions are^ t^en 'itom the U,S* Bi^au %f the 
' Data Books 1977 the identification, of rural st 



th^ir respective peroent of populations outside of central cities 
. Politics Un the Rural States i People , tertian smd I^pcesses by 



^^¥erTOnt 




Jed fi^om the stratified random selection of states 




r^lU!EH TO MCH STATE'S 
MNDICAPPm Am DISADrairaA^^ CONSIZ.TAOT 



T"h e Un 1 verslt5f p f Verm ont 

Df«'^RTMENTOFVOCATldNAL|0U&^TION&TfCHN0tQGY 
COUlQf OP - .lajGULTURi, AORiCULTURAL ENQlNEiHlNG BUiL 

BURLINGTON. VERMONT 05405 




February 3, 1982^ 



' . . = ■ - ■ . I ^ • . 

_ . L ' ■ ........ . . • . 

SRscial Programs Unit 
Department of Public Instruction 
Dfvislon of Vocatlon^^ Education 
Fducition BulTdlnqs " 



Dear 

Thank you for favorably recelvlna my phona inquiry yesterday 
about information concerning \: : (evaluatior^ 

secondary handicapped and dliadvantaged vocational education 
programs - 




sity of 
atlonal 



As I indicated to you in our conversatTonp the 
Vermont 1s under contract with the Vermont Division 
Education to develop an evaluation system for sfeconc 
and disadvantaged programs. Our effort Is to develdp a twoTevel, 
SEA-adml nis tared s evaluation system that can be r 

1) Integrated into Vermont's regular S year cycle for 
evaluation of all programs as per the 1976 evaluation 
mandates; arid, 

2) used as an Independent Instrument In- the evaluation of 
an individual handicapped or disadvantaged program at 
the discretion of the Vermont Special Needs Consultant. 

When we have completed the evaluation system our project develops 
for the Vermont Division of Vocational Educations I will send you a 
copy of the document. 



I look forward to receiving 
disadvantaged evaluation materials, 
cooperation. 



handicapped and 
Thank you for you help and 



incerely. 



dfm Frasier 

Evaluation Project Coordinator 
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JttSilf'Kilyitlfln, ' : 
EMI Enluitisii Tlir 



' * ' yiiiiS Ltfl Avilspiii.jnitpyiiiits 



JitiraaHviiyillsn by in "iiifgFt* uijui nilthiP MA of IE* dtvalopsil 



Jodipindiot third Fifty EyaldatlDr 

. ... . ' ■•■ ' 



dtliif 



D *i yiid In evaiultlcin of dliadvarita|fd prograii 
fi M yiid In ivilyslldi! gf handlcsppid pr^srii 
H yiid In e^alyitlDii of Mtti pfogfants . ' 



^ luESj M mM\mtuUi\n m ysid" tp verify hQ^ 



Olsadvifitagid 



Ifrittefi iurvay Gf stydenb 
ti 
lit 

' ; ■ t. -ffitlofialfa^yity ; 

-pypil syppt'pirsrt 
■r. : ^ ' ^ *prsgfi!ii idvlsdfjf EOsilUii 
*dtli|f ' ' 
PfFignsl JnlirvliHS with^stydinir. 

J- ^adiDiniitfatDFi 

-vocational fakit^:; 
^pypll lypport persopl , 
-Pfogfai adyliorycWiitMe 

'Otill f 

tevliti of studiiit ^fi^Dfdi : , 

^ElstKorss - 
-drdp'pyt fatlar ' 

of LEAf A andTprplffr^, 
^GraiityicatloniP): ? 
4fid of yfar pFdiran fipjrf 
^othir ' / - - 
' Reviiwof SEA ^CDnsultai)t1y1i1t Fipofti 
, ' ' [ ^OBfllifi^i fipofti 

. •HinlM pfBpiii stiiidif^ 



?."s iJiiCLEffl . fio Respgfiir^s Hot pFtlent INw^ 



» M iHluation itfaiijlii in ysid to mify tbi iffiitlveniii of 



MM syrviyef -itydgnU ^ 
-pafifiti 
. ; ^ -iWnlltratQfl 
' •vQciEional fawlty 

':}}^ ' h =pp11 sypppFt peFSQnBil 



DLij.dyajjtiSfd Hatidleapptd 



It 



I -OtlliF _ 



^itaifilitpatoFi 



^4' iyppBFt pi! 



'pFogriis adyjiory Qdnlttli 

iOWlF ' 

JiviW of Itydint =P|ggii" ^ : ; 
^tiii scsrii ^ 
' ^dpop-oyt fitloi 

' ^dthiF 

tola* of Ltt =1 add I yiaF pUni ' 
-OranUppilGatidn (Pj 
: ' ' *lnd of yiir pFOgfini fipoft 
^thiF^ ^ 

SmiW of 'teyltant's vii 



-CDDpllan^i FipOFtl 
^H{fi{ii!Ui!i pfogFliit stindiFds 
''othif 



K*riEE J nUflM lip feppnii ■! it priiint in te^ 
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[ followdjig ttoee ortteiia TOre astatollehed by the 
reMarcter for deterairdng thi utility of re^ew#r*s ectmentsi 

, 1. Only ttosa revlevjer ooiTOents and/or su^estlms whlah V 
□lewly e^anded the focua of tte stii^ were not added 
to tin Inetrimente. Tteea were recorded ae "BSK" 
(Beyorwl the BQOpm of the Study) . 

a. All ad^tlons and/or modlflGatlonB eug^sted by each 

re^d^eTOr would be ^dad to tte COTtent armlyala 
: Instruments, •. ' .. _ . . . 

3* Only thpee delations suggest ad by the actamal Re^rlew 
Panel wa^d be axcluded from tte content analysis 
InstrLmients , 



IMS appendix has been divided^ Into thtea seotlOTS: (a) 
First toterml Review; (b) Second Intamal Re^awi aad (oX Bcternal 
Review Panel. Each section has a aimrary of Ijidl^dual reviewer's 
coniiiants and a sta^TOnt of action taten by the researcher In 
responaa to the re^awer's comenta* content amlysis 

Instruments used by each ^ouplng of ravlawars are presented at ^ 
the end of aadh section. 
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, Goiients of Rivliwer 11 ^ / ^ 


. Mlon !teken 


■ NoConnnti 





H 



c 


■ CcmntB of Reviewer 12 y"^ 


Mlon Tikin 


/r 

^-Msnt of queitions m unclear and wording of queitloni seiid iwteti ^ 


All quiifetoni were 
reworded 



Cdiirinti.ofMewerl3 ' ' ■ :; 


Action Taten 


Uncertto Bboufc' til iiitint of d^^^ 
■ 'Wm"pstioni Vv ; ^ . 


Coiiiint noted by 
reierter 



ERIC 



What types of evaluation strategies ara being used 
by the SEA to evaluata tha £linn1^ PrQcass,ai of 
programi and services designad by LEAs for the 
disadvantaged and handicapped studants enrolled 1n 
vocational education programs? 



SEflMi8N0swHT mm 



piTp MKdiiwiNS Tlii 1TRUCTUI1E .SF A m mmM mm: 



part of, Of separitrffoi, thi itatiU^Qfi|olni lystp fsf yilyUlng iir ; 
VQcaiflfial^edyeitiaii pFPimi within p^rfodof thrflye year stit^plip! 

/ jilipiFata ffga. ^. iz" ■ 

If lepafita ffsi i how ohgfi sri dfiadvintigid ind/of ^indisippid pFepii 
jnd ler^ioii miyatid by th§ilftl ; 



Jer dtiadvantipd prigrainM servtcis 



III , Are disadviniiied ind hindtoappgd progriis gfid sgrvlj^li eyalyated Mith ^: J 
mn Initwnt, df Hith in^vldyilly disipd inltrysifnU? ' ■ , i- 

w ith Individual I sipafata Initfuiints 

with an Intir'agency ivaluatlgn iflitniniiflt diifgnid 
. ■ in eoopmtian with vsyMonil I spiciii i^^^^^ 

III. Mm thi pfeient ^uii" itatus of the SEA'i eyiluilidn domnts ytiliEid 
for evdlyatipfl df liindillppid and/sf. disidyantaged prograins and lefyjcel? 

' GUfrintly biing ySid without PiylilQn , 

■ - " ■ ^ 

, „<^y^fintly bilng mi M with nvlilani.or ladiflcatloni 

currently bitng ysid but undlF revision 

no'ivajiiatlgf) doMnt Gyfrintly bjlng yiid, byt 
. ' ^thi SIA irdiviloplng one for uii wi^ 

■* ' ' ^'^ , ■' RO evaluation doMt^cyfrgntly being ysidj ngr 
doei the lEA hive plini f§f diviloping one 

n o evalyatlon degyiiient{i) win fumiihid by thi 
■ . SEA for yii In thli itydy . 



m'm the fflijoreviryatlon itfitiiy fsriitr by theitt to evilyatfon tllf ' 
Plann fig PrQeiliei of LEA PugfiM and lirvices for dliaavintiged and handieapped 
.-veeailonal edycation itydentsl. x ^ " " = 



Dliidvanla([ed Hand! gapped 



■'LMMNvalyatlgn 
. yilng Uh divilfiped inltfyinents 
■ ilh Self'eviiyalieii ■ ;^ ' 
uiifig UA deVelopid iMtrpents 
[xtcfnal EvaluatlBn Im ; ^ ' 
; ysing M deveieped Initry^ ; / 
.". Jxternil JvalyallonTiaffl 

using ^A dovijfijjed lnstfuiijenii • 
!xUfniI'eva|ynion;by an '^expirt' 
yling SEA devetdped inUrufiiinti ■ 
: htm\ evaiuatldii by 111 "expert"; 
ui'lng LEA devilaned InftrufUDnti ' . 
"Indepdgnt Third Pdrty [valyg^ ■ 
y^lngfleftherlMorLEAinstruraenti 
Eyalugtjgn by State CinsuUant 
\ .yslng lEAJeveiopid initrymenti 
JyaluatiBn by State Eonsyltant . 
j\ .u|lflp p devoloped lnitrponti 
" InfoniuiUviluotlBR by State Censultant 
= ■ using neither SEA' or lea: instinjinentJ 
/Othor . 



KhitM tlie iyiiyatton itfiteglii uiid by thi llhi to verify plMnlnf proceSiii i 
t§ diterilne, tM affirtlveneis of procedyrti yied in planning. proEessis? . 



Written suryiy of - itydefiti ; . . = 
^ parent! 
. * idfolnlstratofi = ^\ 
^- \. "V . ::AVQcat1onaLfkyIty .J 

• pupil lyppflPt pe'rsoiifiil 

S , / ^ prppfft adviipry ceplttii 

' " « ithdr^ 

PirSBnal.lntervi^S with ^ lEydenti 

; ' :x = parents / ' _ 
. ■ ; , . ^ adniiniitratori 

= ' . ■ ■ vocitienii. faculty 

■ pupil luppdrt piripnnal 
^ program a^dyj lory comittee: 
V ^ othe r - ^ • 
Review of itudinl. • recordi /: -v. "~" 
^Jtoit icorei 
iirop^Qyt ratios 
- ■ ether _ :\ 

. Review of L!A:> 1 and iyoar plans 

Oraht Application (IFF) 

* End of yoif prograi report 
5,- otiier ■/ . ' 



led 



Review of lEA * Cbns;t]tint'slistE rGporti 
tapii^nEo rupDrtl 
' Hininiyi^ prpgrp lUndipdl 
V . * fither / 



O il 



J 
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li BofUhe SEA tvalyjU tht nied fsr the pfgpi?) er sifyigi by asieislfig ^ 
thf LfA^effgfti b idintify arid utilii^^ 

tyitplllii the UMqI and cwntiy for hindklppid I dii^dvintaiid itudinti? 



JMffSl yhateyilMtlgft iiritagy tmii m mth^ thi SM te de 
tha LEAU efforts la Itentify and ufllize lyih fisourcQl k 



IF 



Ilk |§lf-ey|luillQh V 

uiingtSEA divelgpad InitFOknt^ 
LEA lelf-tviiuitign . \ 

ySingSiH devalopid initfyfiiintl ' 
toiial [yay Uon Tiinf^ ^ ' . 

uiing j^Eidevikpd Initfupti . 
Ei]ibrn4l EvilijatlopeM H - 

ustiig LEA doyela^f i|isyiientl ^'''r-M^^ 

using;|EA develQpid.j^iifynignts -^J 

uiliif llS^yilepid inslr^^^^ 
'Indepufld^t Thlfdiartylvijultton 
._ using ngfttef lEA of LEf^ioUwntl ' 
Evdlujtioit by Itite Consultant ^ 

using lEA-^dayiloped 1iist«nt| - ■ 
. EvaJuiitlon State teuU^ht-; ■; 
■ uiini LEA devc)opierifl!(ruiiidnti 
Infornial Eviluatipn by ItiitG Cdnsyltiftt.? 

ysing fiiithir SEA or LM InUfuiiienli; " 



V^i yhai eyaluittorr itratiglai ire yiid by tiia SEA to 
\~ lo'ldQntify ind ylillie sy^h jriiBUfEii and lirvke^T 

yfitEirt lyfvfy of ■= Itudents.. ; V ~ 




lA 0m 

Id"' 



^ adiiiniitfatpri . , 
vyecational faculty ■ 

- prolan adyiiory Mjttii 



r parentis 



Review of studint 



^ aflffliniltfatDrs^ ^: ; 

* yecational. facut^ 

* pypii Support perionnil, 

- program adyllQry riplltei 
other . ^' : ■ . . ; , 

toit scofil ; ' , 
= drog^Qut' ratios v » 
= other ^ ■ . 



A. 



£lBltltiateyaluattQnitpati^^ " ^ 

if fg«(veness^ of proeedyrei usid to Idafitify lych mo^fcei 

. ' ■ Bliidyantaggd HandieappeJ 

Mrttten lyrvey of ^ itydifiti " _ " 

= parintr . ' . 

- admlnUtritOFS ^= 
vocitionai fieylty =~ — 

M / = Pypll support pifiQnnll _ ■ , 
■ pfQirain idylsory cdmilttii 
. othe r \ 4 ^ 

^utirviiws with - Uydghti " ;~ . ^ * : . - 
'"^^"^ ^ parinti . ; \ . . j"^^ 

\MdiTiiniitritofr , \ ■ y '~_ ;, 
,% . Mscattonil ficylty ~~ . 

» pupli- support parsonnil ^ ^ ^ . 

■■ . * progriin advisory coeniiu 



Jtvlew of LtA • T'and l yiaf F^^^^^ ' 
V' • : V =- Gfant Application fiFP) 
• End of yoar progfdfn report ; 
• - other . • . V 



'■' Review pf SM.« ConiulUni'i visit feporU 
■ . . ' . Cmpliihqe fcpofts 

Hininp prnoriifi lUndirds : 
' " fl(|ii;r _ 

Thi kk dMS-ne't iiek to vaflfy LEA iffctti. ^ , 
to.' Identify 4 «tniie Mrt'fuioyreel S lervIcK 
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Revfiwpf itydifit 



• recdrdi . 
> test icores 
*^rQp-oyt ritai 
■ other 



feviiw of LiA":l andiypar Piini 
_ . ^ ^ - Grant Application (WPj 
■ ^ ■ ■=:ind:sf 'y&arpfdgrlf^^^^ 
^ dtiii r_ _ _ 
Mm of SEA - fe/iiyltant'v visit reports 
=----.^-Go!!ipIlinct reports \ 
. ^ Hinim progfaiii stjndardi- 
. - other "' - 



Thi lEA'dooi mi leik to verify thi ' , 
affictiv^nill of prgcidyres uiid to \ . 
^ idgntify Lytillii luch risgyrcii k 



. mimmmmtmmmm i 

y '- -^ - - 



4:::- 



- i 
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h Boss the SEA ivilijaU how thi tiandleipped or dliadyanUgid studefit Is 
\m\f\M as needing iirvlcei In irtir to iueeeed in a vacitienal 
educatTsii pfagraiii? . 

, ma lyiuation itritigy fdnpsts are usid by the SM Id detifilfls 

■ ■ how studiflb in RHd Qf lifvlcai iri i^^^^^ 



Dlydvantaged Handicapped 



iniS i Ml ivaluatlQn.stratigies hn yiid fey thi SEA to vgrlfj_ihi 
effegtlyentls of pfpgadyfei ysed to Idifltify ity^pFtsln need 
oF spSTiViifvieiiT" 



UA StlNyaluatlgn 
using SEA divilgpid initfuiiients 
. , LEA SilNvaluitipri 
\ •. iising lift develpd ihitrunffinti 
■ ExCqfiiil EyaluitlDn Tiain 
yilng iEH develdpid instfunents 
Ixtifna] Evalyatipii Tiam 
. yiing lEA divilppid Instfu/nints 
Ixtffnai eyaluitipfi fey in "eKpeft* 
usin^iEA divilppid Instmnientl 
Extofnal ei^aluation by an "Bxpirt'' 
- uiinp LEA deyiiipfd InstMnts 
Indepindont Third Party Eyilyitiof). 

ysing neithir SEA or UA initfyniants 
Evaluitiofi by State Eonsy) tint V 
uling lEA devil gpid initry^inli 
Evaluiticn by State CDnsultaitt 
. yiing LEA deviiapid.instrymenti 
' Inferinal Evalyation by Slate CoRSultaift 
ySlng neithir SEA or LEA initfufflinti 
Other. ' ■ 



IMKi yhat tvaluatian- itfatigfes are used by thi SEA ta verify that' students 
are Eorrietly Identified , as needing sErvtast. ■ ■ " ' ■ . . ■■ 
3 ^ - Pliadvantagid Handicapped 
■ Hrittin Survey of " studio 7 " ■ 
■■■■■ - ■.--.■parents- '■^^■-■'^■■■■^ ^Zj.-' '■ ■- 

- adminiitratQri ^ ^ \_ 
vogatlonal faculty ■ ■ _;, 

-pupil lupport pefiqnnel ^ 

prapii) idyiiory lonittei . 
" ^ Qthl r ' ' _ - 

Personal Intifvlews with - stydents ~ \ 

^ . ' parents ^ ^ 

J ' ' ^ idminiitratDfi ■ . ' ^ 

" . * VQcatlDnai faculty ^ "~" 

* pupil syppoft pifionnel ' 

^ prograni advlipr)' cofiRittlf ■ ^ i - 

/ ' othe r •_ ' " ■ ■ 

Review of itydint - recoriT" . ^ ■ 

- tilt scprei . ■ ^ - . ' ■ 
. ■ . . - drop-oyt ratios _^ 

f * pther _ , 
Piyiiw of LW * 1 and S year Plans 



■ Disadvantaied Handicappid 



. Hrittin survey of - itydents 

- parinlf 

: « Idnliniltfltarl 
= vpcational ficulty ' 
. pupil lypptft pirionnil 
r - pfQpffl Idvliary cOiiittii 

- Pthi r . _ : 

lifsonal jntervfiNl with - itudinti --— — 

■ - parents 
= .adminiltfatofs 
* vgcitiQii|l facylty 

- pupil support peripnnal 

^ program adviiory conittii 
^ othe L_. 
Seviiw of Itydint « recprdr ^ 
• - test scores 
. • drop'Oyt ratei 



RiyiiwDf LEAO andSyeif Plini 

- Orant ApplicitiQn (WP) ^ 
^ End of ylip prpgram report 
' Dtht r. , . . 
RDvfewef SEA * taiultant'l visit reports 
^ ' CQHipi lance report! . 
^ HifijfDum prQiram standards ' 
' other ' 



. Jhp SEA deei.fiotiiik to ifarily tM. . . 
iffect1viness"or procedyrii yiid to" identify 
itydinti In niid of ipeciai services 



mim mmm cohtent ahalm^ 



Orant Application (WPj 
= End of yeir pfggram report.' 
[ ^ other 

Riylew of "^Csniyltaiit's yisti reports 
- Cofflplianci riports 
' ■ = Hinjiutfi prflpiii Siandardi 

^ other 



The SEA does not seik to verTfy thatitydeMs 
are corriCtTFltotjfled is neiding Seryicil 
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85 



0 



3i the KA ^vdluni If fiftiy thQie hjftdicapped S dliadvyfsU^ijd mmi 
■^. jfJfKi Wfiji evalunle^ mm foraits ars uieibf the SEA te dnemlni 



JO^i Hltai.ey^iyatlafi iitmm are used by; thg Sti Id mU^ ik 
. iffoGtlvengii of pfeiedurll yHd ta insyrt iha only sach iiydeflts 
' iri.iervid? " " 



using UA devQlopid InH.rynisnts 

using SW dijveloped iiistru«i 

■ Ext™),[yd)ydti§n Teil 
;• uiing LEA dgvilopid instryiiinti. 

External evaluition by in 'expert* - 
' \ uiing SEA deviloppd initfuiiiinu 

IxterniJl ovaluatioR byjn "e)(p(^* "' 
^ using LEA devs]op§d Instfyiiignii- 

Indt'iiendcht Third lirty EvJuatfon ' . 

: :li?5lyHiQn by Sute Cpnsijltiht, ' 
yiing devtlBped inUryjiienU : 

■ halu^tlon by SUU teulfant 

■ ■ Uiing LEA devqlqptd ifisLruiiicnU : 
' , infofnia] EvaluUlsn by SUtt Consul tsnl 
= * uiing niilher !|A.gf LM instrufnents 

Other 



inil; Nhat iviluatlon itfitiglis iff ysid by thl iEA to viHfy thit only 
sludents neiding lirvldii irt lifvid? 



..Hr'lUtn*syfvi|ef '^ Itudsnts 
pafshls. 



= ad/njnistfitdrl ^ ' , 
^ vocatianli .facylty ■ 
■ '■ - pupil sypparl perssnnil ■ 

■ ' . ■ progfain idrisory tm\im 

^ . = . ' ^ Other ' '. 

PlfSQnirintifvliWS lilth^ students - ^ 
p.parinti ' 
; ■ id-fiinlitfitari . 
'y> vaEltiona) f|Eulty 
• pypll suppoft perspflnil 

^ propmidifiigry CDiilttea 

■ '. ■ - . ^ Bthe r. ' 

Review pf ilydert > reeofjis "~ 
. ^ test seeres ^ 
,-:>drQp-Qyt pates: 

■ QthDP 



Jfftten supvey..ef --^itu^^^ .:Vv; ; 

• parents ; \ ^ 

. * idinjnistfatsfi ' 
vdoatiDnil faciiUy 
- pypij support plfsfiniitl 
prDgram advisofy eoiiittii 

Intefvliwi with ^ studofits " 

• parents 

. * adminiitritQfs 
' vacatienal faculty 

• pupil support pifiannli . 

« pro^m advliery ceiiiltel 
Qthep 



Oij|dvaf)taged HindUappcd 



Riviiw of LEA - 1 ind S year Plans 

■ grant Appiiqatlon (RFF) 

^ Ifid sf yiir progriiii Fiport 

Mmf 314 * Xoniuttant'i visit fipmT 
*Csiiipllan£i fipsfti 
= HinifiHjiii pragraiii sUndardi 

■ other ^ 



MlEA ^sasM iSli^ 
iffeetiveneislT procedures usid ta iniyri 
that only handiapped 'i Diiadvintlgod 
Stydenii are served 



tmm mmm mm mumi 



Review of studint 



' fo$t icsres 
■ dfop-oyt ratloi 

' OtllCf 



Rivlew of LEA ■ 1 and 1 year Plain" 

* Orant AppllcitjQfi (SFH) 

• End of yea/ program report 
■ other ■■ ■ ■ 



Revlfw of SEA * Consul tant' s~vist f reports" 

* COfiipHanei reporti'^ 

* MiniMi prograRi lUftdaf^ 

* other 



The lEA da|i not sells t&^verify that only 
Itydeiitl fiiidliij lef^leii. ire lifVid 
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m m n evalu^H? how tlit pnifjniii se^ke iiiesses Ihi mm needs df 
InJWiyyill pfogrjit) Of lery^t p ^ > 

ifJKi m fviluiitieii itpatejy rtniiiti ari.uied by tji SEA te deiinino ■ 
tetf 4 itydiRl*! ynfflgt neidi |P| idintifiedf, " " 



Dllddyintaqed . Handicapped 



LEA Self 'iViiluUlon 

using SEA dtvileped fnitryiiiKjts \ 
LEA Self*eMlyit|g/i ' r 

uiinfj LES divtlapdd iflitfufiieRl! 
liiernjj Eydluitlon Tiini 
-. using ilh imhm \mimtnh 

using LEA deviloptd instrynients 
UUfn^] m\mfm by an "ixpert* 

yilni) SEA dDVilopid instruments ' s ^ 
Ixtspnai ov^iuatiQn ^ an ^export* 
. using UA develppoj InsErui^^nts 
Indppondent Ihird Fifty Eviluatigp 

using neither SEA of LEA Insirpinti 
Eviiiu,ition by Mi Censyltant ■ 

using SEA devolepid instryjiients 
IvaluJtjon by SUte CBniuifant- 

tisimj LEA deveieped InitfynehU 
Infoffiui Evaluation by lUU Csniyltani ■ 

using fiellliif SEA Of LEA InstfuniinU^ 
flthof 



|FJ|: yhsE evilyatloD stratigill aft ysed by Ehi SEA to m\f^ M I. 
Itudent's yfifliet fiiadl ire eorfictly Idintjfiid? 



lfM \ yhfllivalyaiian itrateglis m used by thi lEA is verify thg 
iffeeliyinlis sf pi^eidufii usgd 
/ stydintf , 

■ Diii'dyantigij HjhdUapped 

Ifrittensyrviyof-stydtnts ■ 

.-. ' , ^mm% _^ 

^ idilnistrrtars • " 

i ^ • vocationil ficylty ^ — . 

)fl * pypil luppoft pefsofin^l " ; . .. ^ . 

\ • • progfaui idyisory cDnmittii ^ " ~" . 

■ ■ , ^ stho r' _^ ■ - ~ ■ 

NflQrIlHntefylgwi Willi - ilyden^^^ \ 

"*parenti . . _ . ■ 

..-^ - adiiilnlitritQii 

^ vocitienal faculty ' " 
' pupil syppsrt pprsonfiil " ■ . " 
• progfini idvliDfy crtttfl 
^Qthi r - • ' ■ 

Hiyliw of itydinl ^ reiQfdi . - - = 

^tastscDfei = )3! ^ 

^ drop^Qui ratfi ^ ' . ^ 



Mriutn Survey flf * siydenti . 

■ ■■ ' ' > parents ^ ' 

= idiiiiniitratsrs 



Glsady^iitaood 



SiifiiWfif LEA- I and i yiar Plans 

= Ofant AppiieatiQn (RFPj 
' End of yeir pfogran fipsrt 
* Qtiier . ■ 

Riyiiw af JEA • CpnsulUni's visit peportl 
, - CofupljanCi repoFti . 

- Hinlfiiy^lfdgfain itindirdi 

- fittier . ' 



^ ■= vocitldnal faculty 

- pupi! suppgrt pahenncl. 
... p progrdni advisory coinnittei 

• other ^ ' 
Pirsonal fntefvliws «lth-^ studenii. 

j . pirenEs 

. ■ - adilnistratofs 

- yocatlQi|ai ficylty 

• pupil iyppoft perlonnal 
propR) idyfsory cortUgf 

' gtiief 
of itydtnt % feceris 
■ . - test icofei 
' . - drdp^dyt Fatlof ■ 

• other . . 



Ihf SEA does notiiek to viflfjf the ettectlveness ■ 
'of procedyfis yiid tt. Identify Ihi yfiit mids ' 
Of stydents 



■Rlyiew of LEA - 1 and 5 year Plans 

' Grant Aiiplieatlon (P) 
* End of year progfaiti repdrt 
• Mthof ■ ■ ^ 

Hei/iew of SEA » Djn^ijlt.iriL-VVIdt repgrlT 
» Diif llnh^D riipflfts, , 
. * HlfiliiiiMH prn'jr.ii!i stdndards 
w oiliiT ^ 

Thi S£i^ doei not seek to verify how the unni^t ~^ 
nfiidi of i itudiflt in^ jdintlfiid. . ■ . 



■ 90^ 



Si' M iU ivilyjte hsw pistil Mtd^iible ibgyt tht mitM m 



No 



jMni Khit etfilyAtioii ttniw fraU ire yiid by ths S[ll 15 imm\m 
' ihi InvolvciariL pesplt kneHhdiQ^bli ibout the stydcni 
iff tteiimii\i(i| whkh.i^^^^ . 

Diiadvantsgid i Hind I gapped 

UA SelNvaiuStl§n 

■ using HA divelopid fnitfyinifttl ' ^ . 

ICA igiNvll|j3tion 

. "using UA devtloptd lnitryffiifiU ' ^ _! 

Externjl CvilyatiQn Teafi) , ■\' 

uilng SEA de^ilp^lnstryninU '/-^^ ^ 

• U{mi\ [vsluHlOii Tiaffi ^ 

using UA divoioptd Instryminti 

evslunion by an 'espeft^ = / . * 

■ usiny dei/ilopid InstryinenU 
ExUffiiil evaluition by in "iprlV 

using LEAdcvaiepid IhUrysenti ^ . __ 

lfldepL?ndcnt Third Party Evalyillfifl ' ^ 

ysing fieithgr itA of LEA Instt .iRints . , 

EvaluHion byitiU Consultant * ■ 

using HA diviiopy instfyin^^ ' 
jvilunisn by State Ccflsui tans . . 

uiing LEA duveUped Initfyiiiintl » " ^ 

infornidl [yalyation by Itati Consul 

using neither SEA, Of LIA inUniffl^^^^ ^ 

Other 



ifJii ijliat evaluation Itratigiis are uied by thi llh to verify LEA 
*~ ~ ifforts to inyolveiych piflpli If) the ailesiimnl? 



yrittifl iurvey of- students 

^ - - 

09 .'^ itoinistrators 

- vQCationil faeuity 
, , ■'■ ■ ■ - pupil iuppOft piridniiil 

* pf^pgraiii advisory fdnFilUee 
■ Qtlier • ^_ _ 
Pifianli intifvliwi with * studeSs" r;;^ 7 
i • parcntr ' ' ' 

. ■ • ad/ninistfftors = 
' ' ^- yDEitiona) faEulty 

» pupil support pirssnnll 
■ * pfegfaia ldy|iai^|Sffr?!lttli 
sthof 



Peview of itudinl - trntW 



- test iwrer . 

- drsp'out ratios 
^ ot'fier 



) ■ . 

If m Whn lyaiuatlsn stfitegiii ire yiid by the IIA to veriMhi 
iffietiveneii of procidyrei liiid to Invalvi ptople in tfilS^. 
' iiiisiient which Idintlfiei thi student's yfiel nieds! ^ 

■ , Dj.sid\/intlj!ia 

yfltten iumy fif vstydenls ; 

, - aWnutfitori- 

- ygeatignal faculty ^ 
. ; = pypll support personnel 

= pfogf^si advisory coflfflittei • 
■ ■Other . r 

. Pifioflil (nlifyliifS.yltli * itydeiiii " ^ 

• parintr . 

* administrators ^ 

- vp^atlflnal f^cult)^ . 
■ pupil syppoft personnel 

= prograBi advisory corinifttee ■ ^ 
' ■other. ■ _ __, . ^ 



tevie^ of slydml ^ reeords 

toH lEorei 
^ dfop'Oyt rates 
ather_ 



Revtiw Of IIA - land 5 year Flans 
' -.s Orant ! 

. End of ygif progrp report 
qthir 



Jivlew of lEA = ConsuTuSTs vllit ripgrtl 
- Eompllance repDrti 
= Hinim progfafii stafldards 
• other - 



The lEA doel ^ iHk to verify the effe£tlyeneil „ 
of procedurerusid to invglve people in the 
asseiwl whtih jdintifiei thi itudinl'r mi 
noedi ' ' 



Rlvtt^ of LEA.- jl and I year Flans 

. - Grant Application (RFP) 
=_End pfyoai pgrafTi, 

liview of iW - CQniuTtant's visfl ripsfts 
• * toipi lance jeptfrti ■ ; 
- Hinijiiyrt) prpgrafn itandardl 
' other 



Thi |EA does hot liek to verify thi invol^eiRent 
af jurt peoplijln thi aisiss/neni4_ / 



V 



9 



utm Ifldlvl^yjl paftfdpant? ■ ' 

mtm]mkn %mim Hmu m mi by the iillMstpini. 
hew Iriiividud paftlctpinti Influence thr 

UAiif 

iislno SMd^^tloped initrymiRtl ". 

, yiin^ LEA dovelaped inUfymofiU ^ > _ ■ . 
vExternji Evjlujtisfi TsiR' . .. ' 
; using SEA develgpd initryffientl ' .. 

uilng LEA devoiopqd instfynitnti ^ ■ 

uMflq iCMeveloped inltryfiienU . - . 

Ejitcrnjl cvJlujtioft by jn "expert* , " ^ ^ 

u^inq LCA developed instryi^ieiiU ■ 
Infciiirtnt lhird.P^fty Evaludtion • . 

ysiiMj MuithDf EEA or LEA Instryfinntl ^ 
Evjludtirn by State CeniuUanf ^ \ ;' 

u^ir*j ''EA deyclDpd instryiiicntl ' 
EyluaiiM) by State teuitar\t « 

y^ini IfA dcyeloped initrynignts . ■ 

innniyl !vJ]uLUiof^by StiUe CDnMts ^ , , 

■ uiing ncithtT Ilk er LEA inUmnts ■ - 
. Othuf . . ' ^ ^ 

IQll: iJhyt ewiluatioh stpategles are iiied by thi SEA to 
indlvlijijal pirtieipdnts Influsncsibg frntlon 
respective ph and objectives? ^ ' 



What evaluatjenjl^t^it "el ire mi hf the SEA to virify the 

: , ' .^UttiUmii of proc^di^rei yiid by the LEA ^ith IndlYidual 

partielpanU to pfflvi^i for influenelRi the fennitlon of theif 

respiciivi piii M ^Jlfiectlvil? . - \i j . a i 
^ ^ Disadvantaged J 

. Mm mm of - m : ; ■ „ 

-^par^^ " = 

' sdiiijM^tPitQrr ^ 
^ vQcilj'C^nil faculty 
* ■ - pypir support pefisnnil _^ 

-■ ■ ^ ^r^mtmi advliofy cpmittee 
- - 
Pifiil Interviews ^Ith ^ styd^i^ ni 



lam 



. aaiiiifii^tpatori 
- vQcatl^nal faculty " 
^ pupil support pirisnnil 
~ prof to advisppy conittei 



Mew of itydint. 



Revlewof LEA 



teit; locorei 
drgp^Q j^t ra 
gti)gr _ 

1 ortl 



tlllf . 



■ yrltten survey of stud^nti 
- pafi)nti 

^ ddifllfiiitfatQPi 

* voeationat faculty 

, pupi] luppart perionnel 
.= pfograiii ddyisofy OTittit 
' sthcf 

Porsaiial lntefvl§«i dtfi ^ UuilL'nlr 

* ■ ^ • parenti 

, 5^ ^ • idmlnUtritors 

■ • ' ■ • yocltional faculty 

* pupil lupppft plriflnnil 

' prograiii advisory ceiiDlttti 

* otHer 



ypiP Flans 
Gr^flt ^pplicition [Wf] ' i 
* End 9^ year prdgram riport '[ 
^othgr / 
Rivliw of lEA ^ Imp ' tant^s viiU reportT ^ 
^■CQiii^ir^nce reports , ■ ^ 
; " :" ^ ' . Hinl^iifi^ pirograin Itindards 
' gthgf _ _ 
, Ihe SEA doii not 0k U w\ fy thi ef fictlveneis 
af prgeedyfil m6 to pr&-vld@ for partteipant'S . 
Ififluenclng of their r^sp=^c£ive pU i objictivis^ 



ITS mmm content ifcwiiii 



tevlivj of studint 



^ feCDrds * 
- test scores 

dfop-Dut ratios 
'Otbor 



' Peyliw of LEA - 1 and 5 year Flam ■ ^ 

^ - Grant Application (SFP) 
« End of yeir prograffl report 

i ' oLher ^ : _. ^ 

. f>Ovlew of SEA - inn^iiliint^i vTfif rcpOfti" 

■ ' C6f!ipliiiiLe fcjiorti 

= Hinliii'iiiJ prfl'jfdi sUndgrdi * 

■ ' . . ...^ ^ 
Tho SEA does not sesk to vtfify that Individual 

partkipants. Influence fonnailon of ioali £ sbjiCtlvH. 




i i iif ' 



94 



7, poci ths SfA ivdiuitf hew thi.Mft';n§edi §f pfOifaffl Or strvicosPticipanU 
. ki\mm kmt\m of Ihi LM4:pro|fi^ qp sirvki gdils and flyiEiivei? 

' Indlvldyii paftklpiiit'S :™et ne^^^ 

- ^ of tht LEA'i propffl sipyke gQali I objectives? : ; : 

* . ■ ■■ ■ DisadviniiQid Hindi e^jpgd 

-^.^ V ; using SEA diviloped InstPyifiiph ; j ; : • 
UA SQlNivalyitioft. ' " ^ ^ 

using LfA dovelcpid In^tfufflen^ ' ^ : 

fitOffij) tvilysUen T€3iii * . ~ 

yiing |[A deveicped inupyeiaii^ - , 

litcmj] [vjluattdri Teaiii. r ' " 

udnq UA develops Ins tryfnenU. 
^ \ Estirn^l cvalu^ticfi by 2n *expett^^ " 

vusinq SEA dQ9e]apid inilf^^^ / , . ^ 

• Islernjl evaiuatifin by an ^ciMft' ^ ' . . ' ™ 

us ino LEA developed InUtantr 
Ind(!(iefldi*nnhird Party CvjIu^ = ^ ^ 

■ yilng iioither SEA or L[A inUriillM^ 
Evalyatioh by State Csniyltlnt > 

using SEA deyelopidinitrufiiint^ . \ v , i 

f valuation by State Consul ynl > ^ ■ ~ 

using LEA kvcloped jftitryititnti 
Infgrflial IviluitiOn by SUte to^^ , ."^ . , 

using nflthir |£A or LEA initFuifjOfitJ 



IF YEl i Hhal evlluUloHtfatiglil art ulid by UUh tp i/erlfy ttill 
individual partldpant's unwt ntedl inflyinci the forJMtloft 
of the LEAU pfpgrafn qr lervice pall indvQ^^ S . 

Pjlid^j^ Hand J capped 

;v . yrltten fyi^ey of--studeiiti - ^ - - ^ ■. ; ^ - - " " 

adniifllstraton ~~ , > 

*^ vocational fiQuity \ ' 
- pupil support persoflnil ~ 
» prograni advisory coWttee ^ ■ 

^ .a " othe r ^ 

Pefsonai Intirvievis wUh - students ' __ - 

■ pareiiti ~ ' " . 

a^iifiistratDfi * 

^ yoytionyl facylty ^ ~ 

^ • pypil suppopi pirionnel : _ 

• proorani advisory feittii _ 
» other . " " 
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iMfES r HHateyaluatlpn itrateglfs are used by tb§ SEA to virify the 
" effectivepeii of prosedypes uied by the LEA with parlidpaiiU 
lO pPQvfde for tlielr Inflyenclng the forniatign of the LEA's 
prograin BP. service piU and objeciivis] 

iirlUiiiiypviy of = itydents ■ ^ 
• ' parenti \ :._[' 

= adfntnistritdpi , : 
^ voeatienal faculty. 
■ = : = pypil support persennel 

> pto^m idvliqpy cgnmlttei 
' ■ othep : 

Finonal Intervlmwith.- studonti , ^ 

- parents 

- adintnistpatdri ■ 
vocHisnal facylty . : 

- pypil support piPlonnel 
■;■ * progpafn advisopy crtttee 

. ^ Qihe p ' ■ ' ■ . 
Rayfe^ of itydint - pecopdT " ~ , 

- test icQpii ■ , 

- drop*out ratii 
^ othD r_ 

ki\^ of LEA - 1 and 5 year Plim " ~" 

- Crant Application (RFP) . 
' End of year pfograin peport 

> othtr 

Review of SIA = Consultant's visit repdPtr 
^ ' topliince riports ' . 

* Hinl™ ppQlPll itaNfdi \ 

. * other . - 



The SEA ^gii^^=sfek to verify m iffedlViiiiss 
of prQeedyriS used- to pravido for the partiQlpatjon 
of pfogras participants in thi fontati§n of 
LEA ppopin pll and objeclivei . 



mm mmm mw mi\ 



^Heview'of stydint 



Rivlewflf LEA' 



Rivltw ef SEA 



Picordr'". 
teU scores 
drop-out ratiol 
. other ■ 



l and S yeapPlahl 
Grant Application (P) 
End of year prsgrani report 
othCP ^ 

CofisuVtgWi visitTepFtT 
tatpli^nci PdpOfls 
Hiniiiiyiii pfograi^ Standards 
othj?r 



TheHWoeiMjiek 

uninet needs InlfTuetici program goals I objietlvas 



ERJC 
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Mv the SEA eyiluati hgw ttie itydpnt-| vocational initructor is' * 
involved in the pkiiping pmm which dovelGpl gsik and objoctivil 

JHESl lihH iviiyiUon strategy formats art uiid by the SEH to dQtoyne ^ 
I how tlie VQiitisnal jnitryctof il Invoivid in lycN planning? 



LEA S^lMvalyatlon ^ ^ 

yiinp SIA^devtlopgdMnllmhtl 
LEA SeiNyiluatlQii 

usino UA developed IflltwntI ' 
iUpnyl [yaluatlofi Tceiii 

'yifiifj SiA dcveiopod (nstruiiieiits ' 
Uwm\ [yaly^tion Im 

ulirig LEA deyiidped InltfufiienlS 
him'i\] ovily^tlon by aii "tmrV- 

using SEA devDloped Instfynienti 
Eiitefnal evdlydtion by an "iiipert* 

ulini) LEA deyeloped inUrufrientj 
Induponilent Third Fifly Evalyation 

usihf) noither iU Or LEA inilfuiptl 
EvniihUiQn by lUio Coniultant 

uMnfj H)\ yeveiojicd initrufiiehtl 
EViSJuJtlfln by Statu CohSultiint 

using "LEA dcvoleiid inUr nil 
Ihhm] Evaluation by Stai-v ^nsultint 

using neither SEA sr LEA iiiitrymenU * 
^ther 



IF m 



What tvalyation itratigies aft uiid by Ihi Uh to verify that 
^ the vqeational Initfuctor 1| involyid in tlii planning pfociss? 

OiydyantaQgd Hinjlcag 

Wim lurviy of ^ Itudenti 



adinlnistfatofs 

. vocational faculty 

' - pupil iupport ^erionnel 

- pr^m adyiibry mlttii 
' cttei' ' . . . , 

Perspnai Intefvigwi with - Uudonts 

i „ pafEnEi 

^ - adniiniUratOrl 

- vseatienal faculty 

] I ^ pupil lupport personnel 

" prpgrain advliory gortttii 

- ethor 

- tmrh. 



IF_pj What iyaluatlsn itrategiDi m used by the ^EA to verify the 
effectiveness of prpeedufes used by iht LEA to InvoivQ the 
VQcational Initryctbf in ^lanninf which dgyoiopi tho goals 

* :; and dbjectivgi for ptin| the stydent'i ynmit noedsl 

' Hrliten syrviy - students ff 
^ -parents ^ / 



« adiiiintstrators 
- VDcatlonal facylt? 

piipll lypport porionnil 
• prograiii idylsor^ cmlltii 

Hnm] \f\imim with * mimi 



'Review of itydent 



' parQnl^ 

* adniifilstratori 

- vBcatignai faeylty 
pupil syppeft piriannel 

■ prsgraiii advisofy eQiiiittei^^ 

- other - . 
= re^nrin ^= 

sydrei ^ 

• drQp'-Olit rati! 

= othe r 

Riview of LEA - 1 and 1 year PlanT 
^ - Grant Application (RFP) 
, = End of year pfograni report 
" Qthy r. 

fiiviiw of SEA - Consyltafit'i viiU reportr 
*r ■ Conipl lance reports 
. • Hiflimm program Uandardi 

- other J 



The SEA i_m ml siek to verify the iffectiveneis of 
procdufos uieTto invplve the vOQatlgni) instruotOf 
in pUnniny which developi thi goals and objietfvei for 
Bi^eting the ityd^Vs \imi mil 



Si 

Ravlfw of sludifit 



ReviiW of LEA 

r 

Review of lEA 



^ teit scores 1 

- drop^out ratios 

' ether ; * 
= 1 and E yiar Plani 

- Gra^t Apf^ication (RFP) 

- fnd of year prograin report 
other 



CtifiipM'incu fi'ijOftS 
riiriliiiiiiii |jr(i'jr,iii! ^i'lmJiirds 



Th? SfA daei not mi 10 \f|rtfy the vocational 
lnstryctdr*^nyQlyinie/|t in siich planning _ 



s ■ 



•■ •iferfluop^ pcrsolis are .Involved in the pUnniiif y-Mch Influencos^thc 
< fofflaib(ionhPprsrafflfli''sWviM'MMli 

■\ - ■ ■ ■ . ' 

[Pmi Whit iviluatlon Itritcjrfo^ts ire used by thelM.la deteffflliie 
' hDN such ki^wledgeibii perioni iff Mai In piaiinlrig! 



Han topped 



LEA Self^e^aluitian . /. . 

using devflopilinstmnU : 
LEASolNviiyatien. 

using U^dy^eispod'initfyiiien^^ . 
Iiitorfiji Evfllyjtisf! Tep ^ ^ 

uling^iiA duveloped InUruniiflti 
External Eva|uHl.Ofi*ta 

uiing LEA ycvGlDpcd JnUruiiiCnU . 
ImM] evdiudlieh by "expift* ^ 

uiinq 5EA develDpod initruMfiU 
Eitt^rfiiji evduatlon by "uWfi* 

using LEA developejd initfunienU 
Inddpfndont TMfd Part^EvaluUiOn 

using neithefrSEA or'liA instiiiirinU 
Evdiyaiign by Mate COfiSulUrii 

lisiii^ HA devtlpd inurponti ^ 
Eyalyjilon by lui^ CenlulUnt 

y^ifii) LEA devclopQd instwnts 
Infuniiiil Evdiuotign by Sldte ConsulUnt 

uiifl^ nOithiT SEA or LEA jnUrpintS 
Other ' _J 



IF m\ «hdt ivaluaiion stratejies ift usad by thi SlA t« vlrify the Inyol^n 
of such pgfso ns In the pl anning which hUmm Pmm goall h Bbjecilvis 



yrltten survey^Ql =■ studinU 



- adminiitrUsfi 

- pupil support personnel 

= prDgriniadvliery comiittie 
■ = otter 
Penenal Ifittfviewi «lth * itudonts 
' ' -■ parontl 

^ J,d!liii|liltr3tBrS 

^ . voylipnil faculty 

* pypti support perssnnel 

= program idviiory comnlttES 

- * other 

Review ef student ^ rccBrST^^ 
^ t&st scofgl 
^ drop'Out'ratips 

* Olhof . 



Review of LEA - 1 and 5 yiar Planl 

i . Grant Applicnibn (RFF) 

- tnd Of ytsr pfograiR ripoft 

- otHef _ - 



Revliw of SEA - Consultant's visit reports 
i ^ CofupH^nee report 

5 Mininiuin proyr^'^ Stsndirds 

^ gtlmr 

Thi SEA dots not SIQk to verify hoy* suEti 
penqni ^ Iniolvid In plannlig^ 



oi l HSn eviluatlon itfitigiei in ulid by-thf Srt to verify the effeotteesi 

of pfoc^durGi uiqd to invoke l(noi<i|!dpablf poilfins jn tte planniiig^lch 
Infiuohcoi the fonnillon of propnt or siNlco'i gq^ls ind objective?' . 

Pjia;dvjft,tj3ed Hindleappjd 

Hritten survey of * ItudpfiU ' . _ . 

^ - pmm , . ™ \ , ^ ' 

* adfninlilFatori \ ^ ^ 

. • voeational facultl. . " . ~ ' 

- pupJl luppgrt persDnnei .\ / ~ 

-.prpgraRi advliory §0!mii1ee — « ^ 

^ other _ ■ ' T "-.' 

PiilonaHnUfvie^lS^llh - itudents : ' " ^ ' ' 



^ pireflti 

^ adfllnistfliofl 

» mfionai 
I pupil luppart perssnni] 
^ prografii adyliory Mlttit 

^ other 

Itydint * ricordi ■ " 
' ' itst icorir 
» drQp-Q!it ratii 
* other 



Rayiaw of liA • 1 if^d I year PUiii > 

* Orant Application (RFP) ' 
"Jnd If year prOf ram rernft 

ii « other J / 

• Beyiew of SM = Consuiiant'i vUinepohr 
= ConipHan^e rtpgrti 
» Niniu pFOl^ain uandards 
othe r 

The SEA dpei notifeHg verify the 
effectiyeneisT? pfo^edyires'yied W ' 
\m\n such porioni In Ihli type &f plannlni 




Wha^ types of evaluation strategies art bting used 
by the SEA to ^valuiti thi Operational Processes 
of prograuis and sefvicfs disigned by LEAs for the 
disadvantaged. and handl capped istudents enrolled in 
vocational' tducatldn'programs? 




orlpi • ■ - 



rr T itpitigj? ffimtl ire .uied fiy tte SIS to ditini@e ' 
r ^ ' 9r lifvici ii Ninff WMtid i| 'gflginllly plipd? 



^yeleped Inslrynients ' 

.yatiofi Tnh . 
:^ devtlopid inltruienti 

diviloped Initfymintl 
jvjluitlon by afi ''expirt^ 
u ^^EA developed inltrpenis 
^' eyaluation by in "esipirt" * 
u; :ig lEA diylioped Initfuinonts 
^''Nind^flt Thlrd Firty Evaltiatisn 
sing neithif SEA or LEA i/iHruitiihts^ 
iluattsn by State Coniuitant 
using SEA developed inUrumenU 
.ifaludtlon bytatf teultaRt^ ' 
uiingiEA d^ej&pid Instfuments 
Inforifial Ivlluitiin by Stati Consultant 
using neithef Sf^or LEA Instwiiti 
DEher 



inK; Wliit iviluitlBfi itrategiis 'm usid by tha S|A w verify that the 
ff^m or leryiee Is being ln^lMtid as pUnnidl 

. s DjiM^antaied 

Written lufviy of - itudfnti ^ ^ 

^ par|n|s " 

- adntinlit-FatorS 

^ ' vSQatlonal faculty 

pypi I support personnel . " 
- - pragfim advisory MUtii 

- othi r 

Personal intefviewi with - studenti " " ' ' 

* parenti 

- adfflinistritSfi ^ ^ 

- vdEdtiena) faculty . 

' ' pupil luppDrt pepsonnel ~ , . 

^ ^ Pfograii) adviiory eDnnUtei ~ 

other 

record! ~~ 



. IF Hl\ What ivaluition itniiilii ire usfd;fey thriEA to viHfy the i^fat 
; of prociduris yiid t§ iRpliiint t^f pr^^^ , ^ 



Reyiew of student 



t^*U scores 
drop-^out ritios 
other 



levlew of LEA ^ I and S year Plant 
- ^ fifiint AjjpilEatlon (RFP) 

= [nd of year prograin report 

I * other 

Reyiew ofeSEA * CnnsultantTvlstt repOftI 
' tapll^nce feimrt^i * 
< Htiiiiiiiiiii prO[]r^ni Handirdl 

The SEA does not seek to virlfylf thr" ' 
program or leFvTcg li being Ifflplioiintid as planned 



pliadvantaged 



Wfltten lurvey of * itydonti . \ , 
pifenii \m 
« « adjfiirilitratori \ 
= yOOitiDnai faculty ^ 

- pupil luppflrt perionne) ■ 

^ ^ ■ , - progra!!! adyliory eofnnittEi 

"« ' ■ ' 6thei ^j___ ; .^^ J,.^ 

perionll interylewi wfth - itudent? ' " V 

' parinti \ '\ 

* • adffllnlitritori ' ' 
■ ' ^■yacational faculty 

» pypil sypport penonnei . 
' propni idylsofy conitteg 

, " pther^^ ' 

Hivietf of itydiitt^ = , records 

- test scores 
« drop^'qut ratel 

pther . >w ■ 
Sfyli^ Of LIA = 1 and E year Plans 

Orant Application (RfF) 
End, of year prqgraRi report 
other ' , 
Riyllrf of llA = CQniyltant-s^yiiit fiportr 
Conjplianci reports 
^\nkm proyriin Itandardi 
' : " Other 

Tdi IM dqei not leik to verify the effedtiveness 
sf procidurn uiid to Ifsplanient tbi* pr^gfain or 
lervki 



EOHMlNTlfflEERHlNrtONTENT ANALySIS 



:l.",Ofiei 'ttijfjlvallt^ |hl,«j!tt3( to whjcfrpispli Involvid in ind 'rffectid by, thi 
: '.pfflSfain or Isfvlci ari/iii igr^mint p pif^mM of Ih m^m ar serylct?, 

■'. JFJS:. .Hliit Bjiluatlpn itf^t^y famti ire mii by the M to ditiiijna 
If'tte yivldyali/affectid sr Inwlvid lii tbHfop^ 

,, j . i . ■! • I 



Pliadvintaged -^ HanM^^^ 



IF yEl ; yhat ivaluatlon itfatagiei m ysid by the SEA' to verify tliejffeeflvineii 
\ ^ of prDcadyfii ysid to assess ttia agFeiffiiiit of Indlvlduali wlth==thi pyrp6ii(i 



)l^Jelf*iyi1y|t|Qn 

^ using SEA dfVBlDpid fii^t™n|i ■. 

LEAleif-ivSlgatlon ^ , ^ 

ijiing LEA diyflQped Initryniinti 
? Externil lyilylElen 1u% 

uifng StA'diveleptd Initfuments. 
: Extirnil Eviluation Taai 
' using LEA develgped InstryRiints 
tmm\ m\Mm by ait H%mi- 

usifig SEA dmkpH imtryinonti 
ExU^nal evaluatiofi by an '^expert" 

liling LEA'devilppid iRitruftiinti 
Indapdant Thipd Party Evdluitlsn 

uiifjg niither iEA of LEA lnit«nti 
Evalyotien by State ConsuUant 
^ using SEA deviloped initruiiienu 
Eyaluatien'fey Stati Consultant , 

yiifig LEA deVeloptd instruiiients 
hkm] Eyilultien by State Consul Unt 

uiing neither SEA or LEA instrutiiiflts 
Other ' 



lEJS - Whit ivalyatlpn jtriteiiei'iFa ysid by the SEA'toj verify that the pinpfli/ 
inyolved or affeetid by the program art in hmmmt with the purpoie(s!f 

^ 'Di|fldy|htiqed fiaiidlcappid 
JrUtin lurviy gf - student! , - - — - 

- parents — " 

- adnrinistratafs , ^^ .j^ ^ ' 
' ' ■ - vecitlonal; faculty- ' ^ ' 

. - pupil luppprt perssnnel ' ~. ; "~ 

^ PWafn adv1sgry.«i£!ie. ^ . ^; * 
^ ^ othe r, ^ 

'JoriQftal Intarvlews with - S£udi(^^^ ". » 

^'parinfi g ' ' ~' ' 



Piiy vantaged Handleapgid 



Written syrviy of • students ' / 

= parents 

, ' adininUtrflteri 

* voqationai ficulty 
-jupll iuppgrt pirsonnel 

* prggram adviiqry cQmnittee 
. . other 

Perianal Interviews wltfi - itudentiV", 



i 



'Riviiirf of itudani 



parent! , ' 
idiTilnistritDrs 
vQEltlsnil ficulty 
pupil luppprt pirlonnel 
program advisory coiinlttei 

othi r " 

records 
tell scores 
drop-out rates 
other' 



Reviiw gf LEA = } Mi i yiat Plans 

Grant Application (RFP) 
'i End of year prcgrara report 
- = other ■ 
Reyiiv/ of 3M - Cpnsultant's vlstt feporti 
I ^ Cofiipnanee reports 

" Hiniiiiuiii progrifii itandirdi 
• othor 



The 5EA does not ieik to virlfy th'e offertiveness ' 
of proiidufis used to ISSISS agmnt of {ndii/ldulll 

MMHENTS coficEiifiQ Mm mimh 



tevlew of itudent - record 



.» adiiilhiitratofs 
= voeatlonal faEuUy 
Vpupii suprt pirionnil 
* Pfograifj adi/jiory ESinmittee 
''Qthef ' 



Is 

test KOres 
drgp-put ratios 
other 



ki\^ of LEA - 1 and I year Plajs 

- Grant Appjlcatiofi (^FP) 

* End of year prggram report 

' ' Qther 

* , Review of SEA - teultant's visit reports 

V f ' Cnidpllanco reports 

^ Hlfil!ii!!iii ^W\m Uanflards 

The SEA does not seek to virlfy if thi^^'"^ 
Indlyldyali are In agriqiiient with the pyrpoie(s) 



I toil {hiSIA iviluati tte tg which piople Invplvad in and if fee ted by 
the pr^m .fir siryici aft iwsri of Ihi Sirvleii proi/ldid by thi ptfiFiin or service? 

' ' ■' — ' ■ ' ".• , -f 

jni|ilhit evalyailon stri^ fgiiite ire uied by thi m to diterfning thi 
v: ■ extint of irtdi vital' rawirgn^^^^ ■ ' 



ifIes 



at evaluatlBH itriteglii ire uied fey thi SEA to verify the iffietlveniii 
ppQeedyfii yiid to iiiiii ik uimi of Individuil mmmi 



Djiadvantae|gd' ^ Hafldlcappe d 



: Us Silf-eyaluitidn 

'yilng SEA'deyiiojed instniflenti 
LEA Si'lf-iviluitlon 
^ uiinp LEA divolepid Instrymints 
Kternai EvaluatlQn Im 

using lEA deviloped fnitruffiinti 
himh] Evaluation Tiani 

yllng LEA. developed Inilryiiignts 
'ExtifniUvaluallDn by in %pift" 
■■ yMnj SEA developd Inltryniinti ' 
External ivaluatlon by an "expirt' 

uiing LEA divilgpid InUruineiits 
Independint Third Party EvaluatiBn 

using neither SEA or LEA instrymenti 
EvaliiaEion by State Consultant 

uling SEA developed InUryrnenis * 
[vdiyatipn by Stat^ CoMujtant , 

usiiig LEA feloped inUmnts 
InfoffMi Evaluation by Stati Coniuitant 
\ using nlithlr SEA or L£A Instruitiinti 
Other 



inES: yhii eviluitlon itrattgigi an mi ^ thi SEA to verify thi ixteni of 

^OjUlWiJ^!'? ^Kp rmis gf thi sirvkes prgvldid by the prograi!! or lirrici? 

D1siavintiged \ Handkapped 
Wrlttgfi survoy of - itudinti ^ — — 

^ ■> pafenti ' 

- adfflinistrators ~^ ~ 
^ vQCitional faculty ~" ~" 

- pupiV sypport periQhnal ^ ™ 
= pFografii advisory coitriltiei 

^ * othir 

Firsonal jgtervliws with Mtd^^ " " 

■ mm ^~ ~ 

idijiiWitrators jj^ 

- voeffional faculty 

i ' ^ pupil Support pificnngl " 

' - prograni idviiery cmlttei " " 

= pthif ~ 

Rivtiw'of itudint regordi ^™ 
^ ' - test scores ■. 

' * drop-put ratig^ 



Qjladvantiqed Hjndteappea 



Wfittin lyrviy ef - stydenti , 
y ' - parofits * 

f . ' aWrilsiratori 

* = vocational jfacylty 

^'pupil iupp#t pengnnel > 
prografn idvisDry Mlttee 

- Qthir _ 
Pirionll Intervlewi with - itudenLf 

- parents 

' 1 ^ adfiilnHtratofi 
' vocatifihal fftylty 

- pupil support pafsgnnii 

" program advlsgry eoiniiittei 
' Qthi r__ 
Rlview of stydiiii ricordr 

= tilt scoris 
■ drop^oyt fatei 
' = other 



fieyiiw of LEA - 1 and I year Flans 

Grant ApplkatiSi (RfPj . 
En^of year prograi report 
■ other 



Rivlitf of lEA CcnsultahC^i viiu reporU 
" Complianor ripprti 
= Hlniiyfli program itandardi 



3 



othgr 



The SEA dgas not Slek to Virify the affiEtlveneli 
of proc^dyrei ysid to assiss the iKtent of 
Ifidlvidyari awafineis ^ " 



'cOHm coiEiifiG mmi mimh 



other 



Review gf LEA - 1 and 1 year Plans" 

, , ^ Grint Application (RFP) 

* ^ * ^ End Of par progrini report 

^ other J -__ 

Revliw ^ SEA - Coniyltint's visit ripgrts 

- Cdti^iiancg reports ■ 

» Niniiiiuin program standards 

^ other 



Thi SEA do?i mt leek to i^eHfy the extent 
of InditfldualT^awiriniSi 



4i Pen till SIA iviiuati Nhai iffeet'tlii pfogrtt.er licvlcg js havjni m i 
pirtjclpant's.perfgiiinEiiB Ms/hep vsdtla^^^ 



Kb 



l/hjt ev4lwtjfln,stfitiai fflinU art uiid tji the SEA tMeteralni tlii 
efffct of t|8 prwriiB br lervlci upDn' the paftlelpint'yirfaminttf 



•UiSilMlliiitlan \ 

LB lifNvalyitlsfi " .. \ r' 

, uilni LEA divelpd InstfuiiinU 

. usIn^rA diVilB^id Init^^^^ ^ " 
. .'Extifnal [valuaiftfi Tef . ■ . ^ ^ 
.uilng LEA devflgpilinstruifienti ' -'-x 
External evalyitioii by an ''ej(p^^ A' 
uilng SEA developed initryminti \ ' " 

, External iyiluatl§n by an 'ispirt' " , i " 
" using LEA deye]opid"jnilruB)ir\tl y_ 
Ifjdipindent Third party ivaijuatibn. > ^ . 

^. uilnf n^thif SEAorLEA Ini^^^^ .J, 
"Evaluation by StatDConiultint ^ / ■ 

using SEA developed instfyments -.. ^ 

Ivaluallon b^flte Conluitpnt ~ 
; using LEA dtv|]qp4 instrM .>^ w^,"' -i- : 

InforialivalyafRin by St^te te^ "^-^-^y^^ 'J - 
using neither SEA or LEA iniifyiniftts r ' 



ILyp' l^ha€ evaluatien itrateglil art ysil^ihl ilA-to vlrl^Uhi iffi^t of i 
; ttrpropBi or ienvice^upon ^ , . 

' ; oiiadvigtigad . landlEi 

. : . W\ ttift iyrvey of - itydtnti . . " ~" / 

: ■ . . - - parinti ■ y/ ' 

^ ^ adfflin|stfitsri ' ; 
, " ' vptiona) faey]^ y - 

■■ ■ * * pupiUypport pirionnel . . . / " 

■ ' pfdgfp adviiory cofninlttia ' 
, " Qthe r \ / ^ . . 

Firignal' Intiryjews with = students . - : ~ 



1 



4' 



]FJ^. Hhat ivalyitipn U mmi by tlie SEA to^virl^ tliiiffietivaEiii. 
' < of the prpgriRi dp leryfci ypan thi particlpint' i perfsnnanErt 



[Mm iurviy §f '. itudinti 
* pafenu' 



Diiidvantiged 



4 



^ vpcillonai faculty " 

- pupil syppprt pefsdBnel 

" prograi adviiory EOfflBitttii 
= ;othgr _ j ^ ' ■ ^ 
?mm] int|pyli«s wlth^stMents . ~ ~ ~ 
, * pirenti 

* ■ adininiiirltarl ' ; - 

^ volilignij fifiylty 

- pypil support pirionnii 

* pfograin idyliery coitii^lttii 

. =; [Review of itydent ^ricordi' ' 
^ \ - ^'tist learei ■ 
- ■ . ■ U drop-pyt ratii 

' othe r 

Review of LEA ^ I and I year Flans 

- Srant Appiicatlcn (RFF) 

' / End of yilF pfograin report 

^' " - 

|. .. .. ...Riviitf.flf JEA> Consuitanvs visit reporti : 

■ . ' Csnipliinci riporti 

^.v . 4/ - HiniM pfograffl itandards. 

■ = '^^'^ other . ^ ^ — 



The ilA doii jot suit to virify the iffktlviiiiii 
of tKe program or ler-vUi .upon the paft]cipant-| 
perfomance in a vooitlonii eduction progfani 



COmENTS MNClRNlNfi MfiTENT ANALYilSi 



a^iiiniltfitors. , ■/ 

- vQcationaj ficyUy.. '' 

f - pu|il 'Support piiiQnnel 

- ppograffl adyiiory coiiittii 
i' othir ^ , 

Rivlew of student - recorSs "■ 
: -test scores , 
. ■ drop'-out ritioi ' 
■ , . ^ othe r ^ ■ 

Review of LEA ' I ^nd I year Plans 



Orint Appiicatisfl (RfP) 
-.|nd of.yeair prsgraiii fiport^ 
\% ■ . ' ' " oth^r • ■ ' : . 

' Rivi^vDfJEA - Consultant's viiU tfpm~ 
- topHance reports . .V;^ 

-■• ^" ■ ' ' Hlntai prograiti- standards 

. - . _ ' * ultor . ■ ■ 

-■ Jhe SEA doll seek to verify the affeit of 
the progrii or sirvici upon the partjilpant' " 



ERIC 



10 



V 



ii: Ooil thi SU ivilyne: the ixtint to pgapli InVDlved jn ind affi^tid by . 
: pfdgm Qp lifvici ire-ln apiimnt hUH thi adeqyaey, Qapklty and condition 
r Initi^^tlsnsl mourcis aviilibli for yie by p^^^^^ ■ = 

• '^si Jo . - ■ 

F^jESi yiiat iviliatten Itpategy foiiats are usid by tit lEA tO'ditinninrthr 
iviniillity of sych-fisoyriii to piFtlelpinti? 

: I . ' - ,a Plsidyintiged / Hindjgappid 

LEA Seif^iVfliuaHdn ^ ' =; * : 
uilng SEA dg^glopid Inilnsni^ 
■ ^ LEA Wf-iviluitl6ft. 
■ uitnHW davilflped fnitpyiiii^^^^ 

; Extirna) lAluation Tiiiii \ 
' using lEA dii/i]ppii tnitriiiinti . 
Estirnal Ivalyjttsn Team / . ^'.^^ 
using LEA divildpad fnUruinenIi ' 
; ■ Enirnaljv^ 

using SEA divelaped" Initnirati^^^^^^^ 
Externii ivaluatian by an "ixpifl* 

ysing LEA devglopid IfhtFiiinti 
. Wipindint Third Party Evaluaisrt 

u^ing nelthlr SEA or LEA initruniinti ^ ' 
EvaluaEion by StKi CMiultant 

using SEA developid initruinantl 
Evaluation by Stati Coniyltanb . 

using LEA divelopd instaents 
^ Informal Evaluition by State Consultant 

using neithir SEA 5r LEA initruiiignts 



ILllr Jt avaluatien itfatigiii ari used fty thi SK to verify tha axteiit of ' 
!t!g availability Qf siieh resmifgM t^ ; 

Disadvintiqid Handicappid 

uritttn supviy of- - students - •" — ' ~ 

. ' parents " ^ ' ~ 

* adipiniitpators ~" 
= vocatiqiial facylty 

. ^ pupil support perienrjel 

* pfogram advisory ertttii 
= othe r ■ 

Nrionil intervleMS with - itudenB — 

'\ ' ' * pmui 

. ' V ' ... .1 adiilrrfstfatBrs ' ? ' - 

- vQcaUonii fieulty ^ 
* ' . pupil support penannel 

I , V- prbgrani idyisory wiiltte^ 

- othe r 

feviiv of itydfnt r ricsfdi .^i . 
. e * test scqrei 

- drop-put ratios 
athir 



IF %% \ lihit evaluation itfategiii are mi by the IfA to virify thi ei ^ 
sf luoh Instruclionii raiourses avaikbji for ysi by pirticipinfs? 



•OfAd\ 



lajdlei 



' ■ Hfittin-iyrvay of - itudints . ' 

* pirents , " ■ 

■ • . idisiniitrateri 

- * VQcitionil faculty 
, pupil iupport pinoiinil . 

/\ . . = propffl idvfiory cpiiiitee 
' ' othir ^ ■ ■ ■ ■ , 
Ferspnal Intirvlews wlth^^ studonti :■ ~ 
• * • - - - parents ' . • 
, ; ^ ^ ' - idminliiritors 
' \- ^ voeational fa§ulty 

,^ pupil lupport personnel 
- prqgfaitiidvlssry csTiniltee 
othif 



Mi^ of itydint 



^records : 
tiitiEares 
drop^oyt ratis 
Qth^ 



Rivliw of LIA ^ l and I yeir Plans 
; « Grint Application 
= End of year progriii} fipprt 
othtr . , 
HiV jfyf of SEA - Csniultint'l ylsU fiporti^ 

- tap! lance reports 
HinjM progfai itindirds 

- other 



OS 




Thi SEA does leek to verify the efftetivinisi 
of luch relQurcil ivillibli for uie by partjclpinti 



^ CMNTS OTCEWilNa CONTENT ANALYStSi 



Reyiiw of LEA - I and S yeif Plani 
. ' / Grant Appiitation (RFP) 

- End of year prograni riporf. - ■ 
■ othe r ^ 

-teviewof-IEA^-^Coniyltant^ 

- CiDRipliaifci nporti ^ 
- , MiniM pfopiii standards. . - ^ 

- other 



The .SEA does not siik to verffylhrixtint 
;ef avallibility of lyGh reioyrecei tb 
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6* 0©ei m la IVilyiti tti U whtsli ifii pregfint fip iirvjoi ii able to m^t 
.... iUM'mm^fmk^g^m, ^ ^ ^ 

^ JFJSi ^ Wiit^evilyitlon fitpatiiy folafi art ustd by the M lo ditiralna thi 
' iMtint to mMcIi profraui.Df lirvict §§iii ifg fcifni (siy 



IFJ^ JKit ivaiuailen itritgglii iri ysid fe thi SEA to vtrl;^ die iffiGtlveniii ■ 
pf thi .program Qr iirv^' in leetlni atad pmiraiD or sir^ic4. pilfl : - 



^0 



L L£A SilNvilyilien. . ' ■ 

.LEA Siif-avalyatien ^ ' .. '~ 

using LEA.develspedlnitruffllnti ■ : . 

Exter^il [Viiyatlsn to . ■ , 

■ yling lEA devglspid initninigfits ' • 

ExternanviluaHgn im^ = ^ ^ . ■ 

' using LEA divelQpidJnstmnti.; » 
Ejittrnai mlyition fey in %^^^^ ~, 
\ using SEA .devtloped' lnitm^ ".■ ..• 
^External ovaluation by an ''|xpert"^^ f- v " ^ ■ 
; using LEA di^iloped initruinan!;! • •! ;^ V _ 
■ Independent Third Pirty jvilyiti in - :=/ 

yiing niithiF SEA orlEA InitM^ . ' - 

Evaluation by.SliU Coniyltant . . ' ^ ■ ' ^ 
. using^SEA divelspdlnitruiiiints _ . ^ 

; Eviluatten by State- teullant . , 
- using LEA divlloped initfuiuents 
•Infofnial Ev4lyatl9n. by State Consultant ~ 
usini nifthiF SEA or LEA instFu'mintl ' ^ . — 

Mkt _ ■ _ ._ _ : . ■ ' — " 

Mi evalyatloh straliilas an yiid by the SEA to viHfy thi eKtint to 
. whieh pifig flFjifvjci g^^^ biing init? ^ - " 

. Biiadvantaqid . Handi 
. KfUten syrviy of - itydi/)ii ^ ■■ : , 

J /.^ ' " P^rinti . ■ " . 

vadmlnistrators ~ 
- volitional faeulty ^ " . = 
'.'^ u * pupil iuppDrt pifisniiel 

. /" " ; ' progpas idviiory cofinjttgi/^"^^ " y"'^' 

. ' . ^ 6th V_^_ _ i^ ' 
Fifiqgal 1ntirvle«s i^lth * studlflts . ~ 

* pirints 

V ■ ^ ■ ■ * idffliRiitrators. ■ ■ ■ ~ 
, ' - volitional faculty = 
/ ' - pupil iuppoFt pirsannil ■ """^ • 
; - pfografii advisory mittii - 

1 ■ '. , ' othe r 

Rivti»f §f sti/dint * records " " " 

. . . • - test scores ^ 
drop-oyt ritios 



pfradvanftgtd Mik P. 



' , > yrittin iurviy if - itydanti 
. ■ , / " -pifinti' \-; • 
. • • adjplnlitratars 
■ , vocatiotiil faEuU ■ . 
•pupil lyppori pifLonnil 
» preprn idviipry ™itiie' 
■ ■'■ ■gthe ^_._ 
Pirlpnil Ifltirvlews ^Ith ^ itydenti ^~ 
pirents. 
\ * idilnlstratori 
. • ydeatlpnil facyllv " 

' pypij lyppoft pi! onnel 
- " ' ' pf^^j'^in advisory oipittei 
other: _ . 



Riviiw of itudiiit 



tevitw of L|A ^ 



reeordr 
test scorn 
-drop-out ratei 
other ' ' 



YES 



1 and lyiif Plans - 
' ' " ■ ^ Grant Appli^atioii (P) 
. - End of jfiar prggrain repof t 

M\i\m of M - Consultant'! visit feporti^ 
" " ■ - CoitipHanci reports r 

• HinlniB proiraia itandardi = 

^ othi r___J -_L _ 

"Thi SEA te not iiik'td 



_ the affictivinaii 

of the prognni of imki in Dietlhg itited pii 

mim mmm content ANALysii; 



^ . RivliH of ilA * 1 ind i.yiar Piani 
.. / " Orantlpplieitien (RFP) . 

- End of yiar prograiD report ^ 
' othe r _ . . V 
Rtviiw of lEA - Cpnsultant's visit ripartT 
j/ ^ . ^- Cpnipliance reports 

' Hifii™ progrm stlndards :i 
"" Vothi r " " " 

The lEA doii not Silk to verify the ixtent to ' 
^ which thi prpgrani of lirvici goals art bitng let . 
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\.MM, Wm^mMi^tf^ im^m m H ttii sea to ditaWne m^^^ ' . . 
, V, '., ^ tifcsl pliCi dyrlfij thi dljly opiFatifiB if thi pfpgran or iifyi^- 

"■r^i;;. ■ "I'V; 'V '■■ '■ ' ' " ' ' \ -: ' ,vA 

V"-!^-!^^^ • ■: ■ ■• %■ ■ - ■ -^. ^ 

> l.ui|fig IEUiviIo^^^ ^ : . /■ , 

' . Jndipindept TMrd Fifty ■ ■ ~ ~ ' 

7'. JvaJyatloOy ?tati to ■ ■ ' ' ■ V~ ' ^ 
uiing S|A dai/itojid ifistrumenh ^^^ ' ■ , 
.U^ilyatton ^ Ititi to ^-y ' . ' ^ - 

• :4jJ|ing LEAldiv|lipd InitruEiiinti ' . -■ ' ■ 

. ' liJfQpl Evalyatififl by*^tsta Cosialtot ■ ^ " ■ : 

^-. '^:i=f^t&-whici! staff act^^^ llen dayl ^ 

c yH%n lyrvgy of , i d antiqid 

: ^/ ■'r^.L^: -.'^ v: . * idffijnfltfltQfl ^ v> 

. V * pupil lupport plfioflnil 

.t pfograi aWlQpy cortttii 

^ * othe r '-.--.^ ; ^ 
' hnm] Intlfvliyi «lth - studihtl ; ■ C - 

^ ."'^ ■ . -i parents. , ' V ; ; 

^ Y ^ ■ .« idnifniitrators - ■ 

' ' : vscitlofial fiEiilty^ ' 

■^-.{\ ^' : pupT) support pgrSDnnel 

• ' i prograifl idvlsory comittii 



iF^ESs^Hhat iVi^uitien itratiglii api usid b/ tfii IM tQ^irlfy Lhi iffictlneiieii 
: ~. =flf yii staff's NeaHlflg of actually tikii .plici lath diyl / 



o 




^Review of itydent 



f iqords 
» telt icDril 
- drop-out ritlos 
^ other 



Siview af LEA - 



1 and l.yeif Plani 
^ ; ; , _ - Grant %1 Icatlon (RFP) . 

^ - lnd;.of year pragrp ripoift 
" gthi r_;_j _ ^ ^ ... 
RiVii^^ otiEA CsnsyUM^;i(llifeFipOftr 
^ Cnflijiikiieo:Tspdrti' , . 
• ifihtoi pfQjrajf) standsrdi 

Thi SEA doii not lA to viri^'wfiK^'''^'" ^ ™ 
takes place dar§f the pr^riiii'i ppiPatfof!' 



itudenti ^ 
,.V; -yparents 
' . . i \ . * IdnilniitritoPl 
' ■ V " vQQtional' faculty \' 
' ^ ■ . '.pypll support pirionflil ' 
. - prepin edviiBj^ crttltii 

Pinsiiil litftryte with vitud^i,, ■ * : ; 

partfiti . r ' ^ . ■ 
iWfljitfatsri 
/ . ' . •:vflC3£lona1. ficuUy 
■ .. ' P^pfl WPrt pirsonnel 

^' - pragran) idvfsQr^ eontaiitii 

/ >^5tf^r ^' ■ • ■ 

. toliwpf iluto *,reiord|- ^ / / 

' •ilSt SCOFiS : ■ 

. V ; , . -frgp^BUt ratil 



fiiview of LEA * ! and S .yiar Plini 

■ • « fet application (P) 
: . - Ifidlb^eap pfogrlfn ftpofi 



itf of SIA COgiulpnt'i visit riports " 
' ' > £o(i$1i^qffep0Hl 

» Hirilttp^graffl standards 



TliiiEA'ddes jot silk to vaflf TijMfftcflviniii '~ _ , 
■ of thg stiffs recsf^lng of^dili^^ef^^^^ 
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/Bjil ii;eii|uiti W tiji Ltt mm^ and loflitsfi on i centimlsiis mi 
.Mmm ^imkimUMmmikMf ^mti^mfm wlthsyt the ^ . 

r^lttff TOltfifs partldpant'i potintiil to luceW'NlthdutJhi p^griia or siFvici? 



IF lUv yiiat iviluittcn itFitigiai in yiid by .thi SEA to virlfy thi iffictiyinH 
"'" " of staff's mltofing pracidyffi to ditinnlni thi pfliintlil of i pirtlQlpant'l 
: iblll^ to mmi In i.VQ£ittonil pfoifiPi ^itteut tM mf^ ^f iii^M ; " =■ 



DJlldMnftged 



VLIA Sflf^ivaiuiljan * 

using lEA dlVikpid ..InitfUSlntS 

LEA Self-ivaluaMon 
yifng LIA devilopid IflUfPinU ; 

; ^" u^iRg IIA dei/ilcsped Inrtrtiiants 
; 'ixtarna,] |\^lyaticiTtif!i 
• usini LIA devilepid initrpints 
Ixifrnal eyglyaticfi ^ in Hmrt^ 
using.SEA devilBpid' Inilrufl^nts 
External evaliiatlsft byrin 'expirt' 
yiing LW drvilQpid inl W^^^^ r 
' VWependent Third Party Evilun - 
using mikr UKot lU initniniinti 
l^atyi'tfon by Statt Consyltint 
; , using SEA divilopidlnitrufninti . : 
■ Evaluatien by Ititi Coniyltint. . 

using LiAdivilopid inswinli . 
' Infennal tvaluition by luta Coniulbt 
using nilthir SEA op LEA Inltr jingnti 



IF iyalyatlQfi strateiiis are^uiid by the StLto virify tlia 

— " - staff:™itaflfl? of p|^ti£ipaflt^^p§tf^ti3^ 

Hrittip sorvay of > Uydante - ■ 7 - 
, . \^ parents ' ~ 
idinlniltfitOfl ^ 
■ / -.voattOfiai'tolty ^ .. 
: ^ pypll Jupport periphnil 7 
_ . ^ - prggras adviiory cofiittia 

- ■ - . ^ 9thii i_ _ ^ 
Pirienil .Intifvlewi wttli jtydeRii — 

^ ::i"|^'7" *'ad?!iiniltratep| 

^ t ^>;./" ■ pypil^iipftpirioniiil ^ ^ 

proiM advisory ceiisittie ■ ~ 
* gthl F-V ■ ^ 
. Rivlew of student ^ recsWs " " " 
tpst scQrai 
- drop-out ratiQs. 
>1)ther _ ' ■ " 

teylaw of LIA - 1 and I yiir Plani,. . ■"■ _ _ ■ 
- Grant Appllcatipn (RFP). ' . 
' ' ^ tnd of year prograni ropsrt 

■ ^- ptbor " , 

' lovlei^ of SM^ Consul unV I YlsltTdpOFtF 
" [Qfi!pl lance rQporti 
_ * HlnliiiiM profiPfliHtandards 

\ ; ' fjtlj(/r ; ; ■_. : 

> Jhe SEA doii nfit to yirffrtiii'itaff'l ~ ' ^■ 

; ^ flfinltorlng pflirtldpant^ potintl^^ 



Hjfijkapped ' 



^ ^Disadyanta^ed Hindleagpij 



. ' yrjttan iyfviy"§f;-;itydent| . * 

'M , - parints " 

• « aiiiinfetratfirf ■ 
' ^ . = mttOfiij ficylty . 
/ I ^ > pypljf iupport pirionnil. 

- - .^ \ > pr09r1|^adi^l|B^ caimilttii 

/- ■ ' •■ - fltlie rL 
Pllionil Intervliws wftli ^ stydenti , ' 
" ^ . , ' ■ O^rinti 
^ j \ ; -.adiiilFfjitritQri 
■ ' " ' -^vycitiDna] faculty 

' ' - pypjl siflipprt piripfina] . 

■ ;^ ■ pro|fiin idviiory Mittei 

^ ' ■ ' ^ ■ ^ Qtiii r _ ^ \ - 
tolia of itydent '-.nccirdi . " ~~T 

■ \ = ' . y > tilt s*^r^ - ^ 

. - dFop-aut ratis 
, . ' - Qtlitr . 



ii: _ 

TP? - r 

iindicappgd 



:7 



' ^ ■ Riyi^ 9f LEA - 1 ind. S yiar Plans" 
. . . . ' , ^ -Orint AppllEitlon (RFF) 
s ; ■. ' - fnd af yiar progrii?! rtport 

, \ ;.^^:jthlr __ . . ' 

' - Rivin/ of Stt:^.ttniuUint's visit fiparti 
■ ^'^^ Hinipia) prograi!! standards 

TliillA dots not liili tq-virify thi iffactiviiieis 
Qfjtaff-i BBHWaring procidyrii 



IllTlMiWflfig CTONT SNALyns^ 
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ii llaes the SMlivaluate:t|ii ixtefiHa i* 

tk pgrii gr lin^fei billivi thf IniipuGtiofial offariiigs in syillabli to . , . 
, dfiiiyantiiid and^W^ .,, , .^o, 

■ 1! JfESi Jhat iviluatlan itritegy fofniiti in ybd ^ tfe S|A to ditsmtna ■ ' , 

■ ' : ' ^ :. : . -'g Cli jdvantigii '< Han J1 sipped 

.. ". C"'^' i--' ■ ^ ' ~~ ' 

uifng llMivilgpid InstfWBti ^ . V , " 
LEASiif»i^Iuatfge% ■■f ~ /; ~ 

/yilnj LEA divglppid InitmU • 1 i ^ " 

. Irttrrfal' Evaluation Tiiiti. . : / 1 ' ■ . ■ ' V^^^. 

:/ ^ ^: uilng lEidtvelopfd ifiitry^nt^^ /v. \:_ -■ ' 
' IxtirniHviIukion'TM ' :^ . "•"" '^ ^ ^ . l; T 

^; Using LEA dei/aispid lnit^^ • ' . 

:^ ^ ^ - Ixterna!' evaluate ~ ■ 

■ % usin|1EidiVglgpid initryiinU^ ; ' * 
„ ' ^ > Extiml avaluition 

■ ulini lSa devilopid initfURi^^ . ■ . "■„.' 
;jridep|fident Third Firly Eval^ V/ 
• ' using neither SEA Or LEA^IiisL^^^ _■ 
. Jvlluationiy lute Coniyltip^ ■■ i/ '^ " " 
usin^EA dayfiopid Irtsirp^^^ ■./ 
^ Evalyitign byltati.tofiiultant i " 
, ^ ; v^^ling LEA toilppidnfistw^ - . 
i . ■ Infjjrniil Ivaiyition b/iState £on|y)lint ~^ 

'Uilng neithaF SEA or LEA instwnU ■ . ^ . 

' ■ ' OtheF " . . * ~ " 



IF m \ MimMMfiirmiu mmi by the ilA to yarifjf thi extent 

....... .t^ Nhich the-progriM O^SirvIci 1l to s£udinti?.,i.; . ... 

\ . Dhadvjnugjd Handici 

^UrJtEen lyUfty of - itudinti 
^■^ ■ . piffnti ^ ' ^ 

• adininiitratori 

: Mscitiona) facylty ^ t i ]T 

- pupil fupport piFionngl ■ ' ^ jl 
J ^ i * pfograii adviisry copittie „ 

* othif 



IF m \ yhat ivalyation striMis art yiid by I i 'Wku virifylhe iffictl^/ifiiii ■ 
of thi proeitei ysid Id niaki thi pfO|r i ^illibli ^to handiei^id and ^ 
diiadvanUgii itydinti! ' ' : ' c ^^""' ' . / ■ X 



1 



^ ^^sadyaRtaofl ' Mdfcappid 



MHjttin lurviy if - itydints ' 
* parints 
f. f Idiiilfiiltratflfi ' 
' ■. ^catiojial fyuUy. ' 
* « pSpil luppgrt pefiQ. el 
f '::.^ ^ ' prsgriffl advissry c§'"Jttii' 
\:/^ ; ^sthi r . ^ 

Pirsenal inltPviiwi with ^ ity^dinti " T" : 
i ■"; :/ ■ ^^pa^lts " 
V; / _ . aWflistraiofL I 
\ ; \ yag^tflfial fiQultyi 
. .X , . ^ pupil luppoft pif^ o]. 
, ■: \ . : ' - prograi advisory arfl ittie 
' ■ ^ mothe r _ 
> teyiew df ittidinVi;.ric§rdi ; " V 
g , ■ - tilt iGorii , " 
; . dropout fitii 
. othir 



- -i 



Riyiew of LIA> I'lnd ^ yeir Plini 

\^^m Applicatign (^'i^j 

.^- fi^dof year pfopr^pport 

\ :/ >>§thir __ 

:^ Keviet/of ltt - Gpnrultant*rvisit i iOrti 

; " ;^ £ai7fl1|fiq| iipgrtr 

^' • HIM^^ograni star fi% 
■ ^^i ^'^^ ' -^^ 
Ttil iff doai not "leikTa yirify tfie ifflgt: nisi 
of pfoceduni mi to (naki the pfQim tbla 



MINTS coNEEWiHi Went MLysis i 



Perisnii intiryitwi witii - studinti 
- parinU 
* idiinlstratofs 
' fvacatlgiial faculty 

■ 'i -"pupil sypport pirso 

^- progfaii) adyiiopy eo^ittli 
. . pother 
Beyliw of itudint * recofdi 
* ^- tost mm. . 

^ drgp^put Fltigi 
r") : . = Other- 



■ Rivtew of LEA- t and 5 yiar Man! 

' ' ' Grant Application (WP) 
. - End of yeiF program report 
■ . ' other . 

, Sivliw Of lEA - ConiuUant'i ylilt reporti 
. ' ^ taipIlfliiEo reporll 

^ ^ KinitiRjm pr^griiii lUndirdi 

' Thi SEA deii ^ iiefe to verify t¥i eVtant' 
to tdtich thi pfogrin or sefvice is ayiilabh 
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,tpi pregfam or sirvki bsHivi IN Instfuctisfiil jffiHfl|r fl^^ injpirfBDf jatilf 
matey with fdintlfiad stydent piidi?:- > - ■ 

t IFJIi Kliit ivalyilloR itrategy fomatr^ri used iy the SEA ia dleraini that 
^/ ... «rt initructlonal Dfferingi m . 



IF YEi i iit iViluitlQii itfitiglii are used ^ thi to yirify tlie iffieyvifliii 
of pfOEidufii yiid tsiatch Initfuct "iiil offaHngi wltti itydtnt (liedil • 



UA SiiNivaluailafi: ' f 
^ using SEA imhm Instrptfiti / . 
' LEA lilNvllyatlon . ' . 

ullii| LEA diveloM InstMnts? 
hternil Evaluation Tiafn: ^ 
uilng SlA dayilppgd Instfyients' 
imm] EvalyitjoR Tiasi^. i 
• using LEA diytloped/lilitruinintr 
. Ixtihal evaluailQn by;ii^;%peft** i . 

ysing SEA diyelDped initriipiiitl V ^ 
Ixtifna) evilyatlen by in "expert* 7 ' 
, uiini LIA divelgpad InitruieRts ; 
Independeiit Third Party Eviluation^' 
•using liiitheF SEA or LEA-lnitryRitnts 
iyaluatiQn by State Consultant; 
.. -uilng SiA devilDpid instfyininWY. 
^Ivaluatlan by Itita Coniultant . 

uilng LEA divelopid initfurtl 
Infonnal Evalyatisn by State Consultant 
using niith^f SIA Qf L^A initrumgnli 
Other ■ 



. Djiadyantjgid ' HandjjipM 



' tfHtten syrv^of ^ stydents > 
; parents 

» adifllniltritpri . ' 
> voKtisnal facyltv 
' pupil iuppert Jev Blel" 
V - progriin advliory ^^WltH 
"vothe r '. 
Pififinil tfttirvietfS'wUh * itudentS; ' " ~" ' ; ~J 
/ .' ■ vparenti 

aWnlitratfifi 
■ : . \-'vocational=fiCulti' 

- pupil iyppprt pifiOnnil 
, • * pFeiriii) advisory CDiiiiiititea 
" ' ' othe r , ' ' _ ^_ \ 
toiew of stydent - ricordi ' ■ : 
='tist scores ; 
« drep^flut fjtii 
J * ottiif 
Sivjeif of LIA^ 



1 



f . 



JFJjj i Hhit i\faluatiQn. stratigies are usid by thi SH tg verify iihat thg tfftringi 
: ~.^ ayailable are apprQprUtalyjatehid.^itli Identlfii^ itudent needs? .. 

^ Disadyantagad Handicapped 
Hrittift igrvey of « Itydinti ' - " " \ 

• parents _~ ' ' ~ 

■ « adiiiiniitrltori 
> vocational faculty 
^ pupil support pirio 
/■ * prograjfl idvisory caiittii 

/ . ' othi r . _ " ■ 

Mmti Intervievis with - itudentl ~ ~ ~ 
- parentl 

' >; .. ' idfiiinlstratori 

/ , ' - yoCitlQnil faculty 

./ i ■ - pupil support personnel 

- prQ|m advisory coiilttei 
' ■ pthir f 



Syeir Mans' 
Grint AppjiiatiOh (^FF) 
■ Eno of yiar pFogriiiii riport 
- otfef ' . ,1 ■ 

Riyiiw. of SIA = Consultant's visU reports 

^ ^ HiniiTiy|i) prggrlifl standards 
■ >othi f ■■ ' 
/The IIA does ^ laalt to verify tha^iffictiveniss 
of procedureiljsed to iiiatch ibdint n^^^ 
■ ^propriite instnirtlon ^ ■ '^^ ' 

CQMTS CQNEEilfiO MHTiNT ANALYSIS 1 



ievliw of itudent: 



- racgfdl- . 
' im mm 
'/drop-out ratios 

- other 



Review of LIA - 1 and S yiar Plani 

- Ofint Appllcatign (IFP) 

- End of year progrifii:report' 
"Othe r^ ' " ^ . 

Reviaw lof SEA^ ConsyTYahl'i vTiiTTOrfi 
= Coiiipl tinea fipofti 
'^i ^ = Hinimym prograi!) Itandards 

. ■ - other . -. ; : : :_ 



Ihe SEA doii notiaak ip" verify the natch 
of Initrueti^rBasid on stydint neidi= 



^1 
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ilywion laKSn 


Itelete Opamtioul ftooiis Instmint fiuiitlQn: "tea the 11 iiliite whitlir 


. .L_--' 


the ppm BtifT ideiffli iltS i a eiujfiritf^^ 
^ partlcipinte to 'Succeed In their Tiiptlvi vocitlonai pwpmi without' the 
1 proMfSihereifterrefeired to ii ^sratloilteesi W toitlon 18) 


'"^otfrliy™" 
reaeircher 

* ' ' ' ' 


• Mifci OperitilQiil tecess Mmnfc ftieitloni "Doei thi 31 iilite wtethir 
ppli/lnvoivid in and af ftetid bj the ppogriifi beUsva thi Wruetionil 
offirirli ari imlMile to thou diiaflvantiied and hinidappid studenti most In 
niid of the pragPii? (hiMftir pft^^ ae ^emtional teii^tnient; 
QueitionW ' ^ ^ ■ - 


ibtedby 
nsi^her 



' \:' doiientBof ftvliwerlS ' ■ ■ ' > : ■ ^* 


Action Ito 


,l^lete Operational teiSi^ 

■ ■ .—^1- ■■ ■ ■ ' . 


ffetid by ' . 
reiearQhsr , 


Md to Operational frmm Instrument tl quiBtion; "Doia tte SI smluatg ■ ' . ^ 
whete the propni^ Ims an ippropriate teite/pupil ratio?" 


Queition 
addid 


' whstber the propm'i inifcruette has the teacher qualiflcatioiis to work. ■ 

with handiGapped and/or diiatatipd per^^^ ' - - , . 

" ■ ■ 7 ., ^ ■■■■ ■ ' ■■ ' ■ ■ ' " . - i 


™Qiiistion~^ 
' added . ' 


. Add to PlOTlng Process Instrment the queiioni "Doei the 81 emluate 
: wSther the LI has estibiished tlnfilinis for topliinintiii nijor program , ■ 
goals and obJiGllvBsf .. 

. — t ^ / ' ■■---•--.^^ 


§iieitlon - . 
a^ed' 


Restruetiffs Siting of stratep foMt posiibiHtiei to m^ 
potently conbiiitioni . . ^ 

. . ■ . . 1 ... ^ : - . '. 


.Listing . 
restruotmfd _ 


D^lop letter code for piible answera to "If li". pitlons aid ; ' C 
hlgilip with a boidike enc^ 


pdi,diVilopidl. 
inclosid 


Wvlde iieh phaii of thi content ahalyeis ihiet by boxin| "If Yes" questlom 


-i ... 

ftiistions wen , 
snsloied 



' ^ [ . . ^ ' PROPOSED USE OF tHIS IHSTRUMENT ' ' -, ' , ; • ; ; y 

This instrufaenj; wtll.be used for ionductlng a qualitative. content analysis of docunienti used 
by state erfucatlon agencies (SEAs) to eyaluati the iffectiveness of Ipcal edutation agancy ■ 
(LEA) vocitfonal education pRogranis for thif handicapped and diiadvantag student, ^ , ' 



■ - m QUESTION OF THIS INSTRU^^ 

What type^nf evaluation stratiaiei a ulii by SEAs to evaluita QPEtaTIONAL PRQCE^^^^ 

if [EfTocatMiraucI^^ handicapped and disadvintagid student? 



Prepared by 
Jlni Frailer 



, i^mWon dpiiinEi mm the evilyitim of prepn" OPERATOR PfDCllS^ 




\k y« Hid tesi quiltiffiii liU p pliiii 



i 



L criMealiy rirtf K ^ £(1^^ sn the ippr^rtatemii of each 



i; rtilltytj quiitlonB within jhi Wmiint ftiMt; 



2i offtf MiUbl qiiltl^ pu tMrik eOuld bi i^d te IdintlN 



^ iliiiljf .fivlfs and tmmi on thi i^rtprliteniia of uilng 



•lit jis,,, thin" fmat le IM^ OFmTIONAi; FliDCEsaiSi indi 




1. 



De^i tte SEA eviluite whithir the pr^ m bit n ln^liiintid as ' 



tte p i\iiyiti witto picple InvoM In ijia ifftetid b^'the' : 
prepjn m of thi iervlcii previid by tte prsps? ■ 




prepni in Iri ipe!int:yrtth the^id^yigy,^ e^adty and/or crtlMon ^ 
of MMral iiiaiMs 

. , .^^ :M. to jp etouate la hivlf| in.in^gt en thi:^ ^ 

participants perfsmei in Mt repQU^ mtiml idygtlm prspflii ■ 

i ^itteSH iVEluitiwtethirthi.pi^te ^ 
prspiiigsli? ' 

toe thi aa iviluite whitter the pK^irt itif f l^ntlHf i M itrirda 
wliuotually Eiii pte iring the Mly optrntiai of m prsffiii? " 

I m the Si tviMtt j^hithif thi pr^ itaff Identiflei raltofi 
on I eentinuoui be^li the petintiai of pirtielpanti Es luoo^i in thiir 
. repotivi vBcitlcnd propji wlthait tli prcpp? ■ i ■ 

9. te tte BEA iMiate viiEhiP pigpk 

prqpn biliivi thi Instotional offtrljip iiTi aviilali ts tto^ ^ 
dlsadMitapd md hmdieappid itydeiits gt in riid ef thi prsp/n? 

10* toi tli M iviluiti whiEtwr ptepli Involvid in ind iffgetid by Ell 
pr^ billi^ thi jtetruetiaiil" of firlnp availabli m ipprspriately . 
^itohgd.w^th(lda^Eiflid it« , .. 
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irisiiiillfplannid? 



UMM lyaluitlpn itnligy fannali m m 
■ ■ ■ . ■ if thi prapi-or lifylK li telrig ^1 



llHilNyilyiften 

■ yiing ItA imU0 fnitfWnts 

yilng LEA devilopei Instfpinti 
[UM] Evilyatlon Tip 
, ysing SfA dgyelopfd Initiwnts 

yiing LEA diVileped InstfyiDinU 
jKlirnai ivalyatigii fey an ^ixperi" 
, using SES devil.opid Initrynionti 
[KUrnH evalyitlon by in *OJ(pert* ^ 
lislng tEA.divaleped Instrijicntl 
Wependiiit Third Party Evslyatlon 
■yiini neittiif lEA or LEA instryn-Jntl 
WuaUonbyJiitrConiy^ . 
jllfig SEA im^mi Initniiiitl 
ftduatlDfi by Stati ConiyUant r ^ 
using LEA tollepid initruiiishti 
'Itel EyilyatiDR by Hate teyltaiit 
' yiing nsltliir JEA or L|A litstrymanti 



by UlSEA ta diiirnilna ^ 
intid II ariginilly pUnnid? 



IF II 



I Kilt i\falyiti§^ itfitiiiii m mi ^ lhr!|A to virify-thi iffn 
^ of prQcidyfii uiid to linptent thtprtgrii qp iifvieif ; ^ . . 

liliMiiM . f 



Mttin lyrviy of * itudonts , ' 
. : § i , - parents . ' 

- adiiilntUritQri 
^ . vvdeational fmUi 

. , pupjl syppgft piflonflii ^ 
- * pregrta adyiiory cofinittia 

IKS wftli - Itydgnti " ^ ^ 

• p™trf 
« adtnifliifrHori 

, - vQcatfanil facylty >, 

* ptl lupport^pirtonfiil . 
^ progfasi idi/liDfy ertttii 

. ^ a Qtkf_ _ 



f of iMnt 



.ficordi 
' test icprii 
' drpp^Qyt ratiS 
■ other 



^JSi It evilyation itratigiii an yiid by \hm SEA to virf fy that thi 
" ffoirani Qf iiryfci li Eatni tif liintid as plannidl 

Dltadvanfiged H_andkappsd 

y ' (flttin syryiy §f * Itudtnti . . . " - - 



tevlew of LEA - 1 and 5 year tVmf 

- Orant AppIlEitlga 0P] 

- fnd Qf yiar prsgraii report 
^ sthir _^ _ _ 

Eivlw Of SEA - CQflsyiMrrflsirfiKm* 

' EoiiipIianQl tipOfti 
^ Hiniu progpiiii itindiirdi 
Mther 



* adiiniitritpri 
■ ■ . ■■ - vocatlgnai ficylty 

-^lypn iyppoH persefi^ 

- pfograi advisory CCflsiittif 
' "Qtha r 

Persil IdtirvfiMS with • Itudenti — 
^- -^parents ..^^--^^ 
!* * ad^inistratofS 

' ;/ ' ' vocational faculty ^ 

" • pupil lupport jiriBfime] ' 
■ - prograin idvtiory CBitesiuei 

kk)H of Itiidini = fiCordT" ■ .. - 
■ - tpil mm 

- drop'Qut ritloi 
^ \ ' other ■ ■ 



111 1 dcfii not ink to vifify thi affiatlWiii 
cfprocidyfis usid to \^\mnt the prograndf 

Wtt./.^,...:. -..-L..^ .' 



Jflfl^ mmm £OHTiNT_AHALjfSlS; 



Bgyjfw of LEA - i and 1 year Plini 

- fir*int AiipiUatlon {W^) 
» End of year prograiii r^pari 
f Dthe f ' _ _ 

tevifw of SEA - teyltanVi'viilf rSfiof IT 

* £fti!ipTl3nco nprti 

* Hjfilim pf^m stinfiadi 

' filljiT 

ppor iei^ci 1i being liflplipntid ai pi flinid 
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" ' PROPOSED USE OF THIS INSTRliHENl , ^ ■ 

This instfufiient will be used for conducting 'a qualitative content analysis of documints used', 
by state Iducation agencies (SEAs) to evaluate the effectiveness, of local education agency^. ■ 
(LEA) vocatlonil education progriiiis for the handi^capped disadvantaged student, ' / 



i .... . ■ . , 

HAJOR QUESTION OF THIS INSTMENT : ^ 

What %pis of evaluation stfateglei aire being used by SEA^to ivaTuate the PLAIING PROCESSES 

of LEA vocational education progranis for the handicappid and disadvantaged student? 



\. . . ' : . . . ■ ■ . . . : ^ . 

I 



Prepared by 
Jim Frasier 



10 PH0CS3ES. 



tej^^y thiitquiiWWi l^iUyoyp • 

1." orttkall^ nvlgi^ and ementai thf app^rlateneig of m&\ 

' 2, sffif aiditlonl quiiHOT you thiji ^ftild be uied to IttentlfV 

L . Itei tte SEA tmlyati etoi nsdi by thi HA uUllie loollly 
ivall^la hiJidloipd/diiad^yipd res ■ ■ 

i. . Ucis' tHi M g\ilyitf effsrts nide by thi UA pr^mi io Idintlfy 

VDoatisiil itoitlm pro 

3i to ths SEA eviliite wti^^ * 

■ VMUe/il iduoatioi prpiflf ^ . 

i Doll ths SEA gviluiti White tte I|A props aB^ases tin irat need! 

Dots thi JEA ivil^lti whiEbir 

Invelviid in asi§5§i3int aefeivilles thil to identify bte ptydint-s 
ymt nifdi? _ . . ; 

L thi SEA gviinti whibhiF thi wmt niida.af indlvidyil atudinti ' 

?, tos tte SEA eViMta whatter tte ispi needi of Indl^diill itudinti ^ 
, Influanai thi fonnatto or. thi HA pr6gp|B goali ifld ,8bJeEtivgg? 

t . Doii tig SIA evilui^i ^tti r thi iMnt- i vsoationil iiiitnjoior li 
' toM in thi planning 

Ji to thi SEA imluate whithir indlvlduile M thi niidi 

of hindieippi^/diiadvertig^d studenti m inniM In tteplaniiing which 
influineii thi fcpEWm of the M prggpii geals md djjietlvgs? 



I? 



I ti^ divelpd m "If yei.Mthen" inathunt fsnifc to IdinEify Ihi' , 
typii of ivaluatlin itrnte^u. btl^ ikid by SEAi te ivaluite pi^gui 

Ai yeyj^vlew thi qmitlw en the ^poiitt lidt of tMi plpp will yoj pit 

Ip. aiibitMeqyiiWofE within the. ^^^^ 

h ■efitleally nviiw mi emint 'ofi thi o^r^riatiniss of uiin| ' 
tte "If yii,,,£hin" im% to Idiiitiftf PUIINIIJ^ Pfl^ffiSj fjij^ ^ 



3, if apprqirlitii bfffr.in iltimtlvi fomt kv m In IdinUfylng 
tpief i^li^tlm itrati^is using SHjv^^ 



1. 



08is ttii tlh m\m thi find far thi mm, Bf iirylM'by mn^i 

thf LW'j effsfts to laintlfy and yUllie ffMUKfl Ind lef^lceii 

iyitiibli (n the.sM and eenlufiity fo^ hindiEip^el I dlsadyiniapd Uydenti! 

IPjil Hhrt evaluitlon stntigji fsnniti are Uild fe/ ik ilk te ditMlne 
■ * thi LM'I ifhrti to IdgRtlfy and utlliie syeh riisyrm i serylm? 



iffocllvifiiii §f prMtdiiCii yied to Identic sy^h niBypqes 
ind iivlEis? . - - ; - 



e 

N 



using M diviioped Initrynilnti 
UA lelNviluiEldfi ^ 

using LEMeyeioped IhUmnU^ 
lmm\ Evalyatlon Tiini ^ 
^ "Ruling IfAdevtlgpid jnUifymiftU 
.litprnal Evaluation Im 
. usfng LEA de^elopid inita^ 
Extfffiai evaiiiitlgrt by an '^OKpcrtV ; 

ulini SEA im\mi InstryiiiinU 
liternai eviluatlDn by in -espeft'* 
■■ ysing LEAideyiloped lnnryfngnti 
Indepyndifit Ihlrd Party Iviluitlen^^ 

uiing miikf SEA Of UA InitWtS 
. Enaluation by Stati CQniuitint . 

uiIdI sea divilopid Ihftmnts 
IvilyaUofi by St^tlMuiUnt 

using IH developed Instrunionts ' 
Inrml EvalyUlon by Stati taltanl 

using Hi iiliaf SEA ar lEA ih^tryinenli 
Other 



IF VESt Hjiat ivaluitlan itratiglii'iri y^id by thi SEA t§ ^jfllW ifforti 
ta Idenllfy and ytlliie su^h mmmi and iirvlces? 
: '\ ' ■ . Bliadyintagid ' 

. yHttm iyrvey of - Studentr 
, . - partflU - 

■ adffliniitritQri * ; 
a . * voeationai tolty • 
» pupil suppoftpipstanel 

* pfogriRl idviiafy CMlttil^ 

Fifbnii ifitsryjiwi iiltll - liydtnls" ' ■ y~ ~" 

* parents: . ^ " 
i ■ - idiinhfFators ^ r i 

^ - vecallonal facylly: ^ 

^ = . . ^ ^ ^ pupil lypppft pOFlonnil 

- pfDOHiii advisory cofilttee 

* gtiigr 

Sevliw of liudeiit « records 7 ■ / 
^ test scorn = 
i drop*out ratios ^ 
i. ■ fithef ■ ■ V ■ ^ 



Diiadi^ajiiajid HandlQppjd 



Uflttin survey §f^iludgntl 

* parinti ■ 

■ , ; - adiittniUratori 
. ' : - vocatlonil facuity 

. { ^ pupil Support pirsoiinil 

* pragram advisory toiilUti 
r . ; »:Otliir _: 

PifSQfiil Intarvtiwi with ^ stydints " 

* idiilnlitfatflrs' _ = 

■ ratlinai faculty 

" pypij luppart pifignni) 

■ iFsgrar advisory c^lttti 
V- = Bthir ^ 

ilvliw §f itudint - riMrlT 



fiivliw of LEA 



i tilt scores 
' drop'fiut fatis 
■' other \ 
d He 



] and iyear Flim 
Grant Applicatlgn [m] 

' otlier ^ _ . . ; • ; _ 
.EiVifw of HA ^ Coiiiyiunl's. visit fiportl 
■ EoiiipllanEi ropOfts 
' . « Hinin program itindards 
' othir i 



The SEA dpei not iqefe to yerlfi^thi ' 
iffictlviniii of proiedyrii ysid to 
j^ntify.l y{i]lii sygh fiiiyrsei I sifvlsii 



amm mmm cohtent mniii 



Rivliif of UA H and i yiar Flani 
= ^ ^ * Grant llppilcation (P) 
^ Er\|i of year pr|irain riporl 

_..::y.otheri: .:. ... .. :: : . 



/ Sevlew of !EA * Consuitaftt I m\i foporti 

■ toipllanci fDports 

■ Jiini™ proijraifl lUndards 

Thi SEA does not liok to vif I fy LEA i f fofti 
O jj|djntifyiyti11iilyChr(!loyreit 

-ERiq 
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\' "Gaiiints of ^ ' ./i'^^ 

.■ ' . 'i.", i ■ '. . ' ■ ■ .. ' ■'. ■ ■ .i ■ 


AetlonMsn 


Md , to Opitetional tesi litPiient a pition tat \ktlmv tli g^s iM 
objaetivea >plamed to hslp the student sucseed in a voeitlonal propaiii havi 
been liipleiifinted by til pragraiii as / ^ ' ■ ■ 


Queatlon 15 
■ added 


M to ^rational ftwesi tetmint i question about wlitter the pi?enti 
Md appropriate staff participated in tap 
in^vldualizsd propm 

ft 

, . ■ . . ■ ^ ■- ■ 


Question H , 
iddid . . 
. i 


kmi: M Planning Prociii Instniint ftiistipn 13 to fosui on the efforti . 
...nadsbyM!^ 

■ "ptQ^m . -. ■ ' 


tetiohIB 

iwwordid.....^;; 

s 


tolete in toemtional, Process Mot 

' t . . 


• Question IB . 
■ dilitsd 


Klete in ^erational R?ocess Initriiint testion 19 


, ' Qiastion l9 
dslstsd 


_W|te Opemtional teesi Instninent Question fl2i "Doss Vm SI, pluati 
,,wtether the propm's instruQtor(i) has fchi ^sissiry quallnQitloni to woric 
C with H/D vocatlomi studinti iTO^ 

. \ \ ■ ' \ ■ ' ' ' , ■ ■ ' ' . ' ■ : 


N) aetion taten 
, question foMlatid 

rsqutaenli 




■ iiERiLiyiiPm 



, , • ■ CdiiEnti of Miiir II (eontinuid)- = , ' ^ 


.Acti'oi 'Men • ' 

if"; ' . 


Spnd f^t to identic reasons for why a pirtlcu^ eontint aalysii. 
indlcatorwai .not adtoised by tte 51 ; ^ 

■ , ' , • , * i 


M action ^ . 


Idinti^ tin interval' for how oftin^ tli SI addressed a content analysis 
Intcator ^ 


Noaetlon 


EitibHsh m ertent to tMch SI ivaluatlon pmctices hivi besn , , 
established Impliminting eontant ina^ils indicatori 

— _. ' ' ■ i ' ' '■■ 


iNo action 
■ {BSTS) 


■ M to ppimtlonal PfOQisi Initriiint qiiistims which' focui on iither thi 
, SM had eviliaated thi ^phpriaterisB and iffectlviiiss..of IM's due 
process proeidures ■ 

b ' ' ■ . i 


. No action 

im) ■; • 

■', ^ 


M to (^rational, Frociss Mnient quistloni which focus on in-EsMee 
nieds activities prtdid to teachers - 


No action 
(BHS) : 


^ Add to PlaiiMng ftttceis Ihstnient a question which focuses on the piling 
procisses usid by IM to estmsh ind^^^ ^ 


NO'-actidn 
,lBSTSr 


Bip^quistions in Plannini tocess Jiistrurenfc . to. lintifir emluatlon 
, practicis usid by Sl'i idth itudentis piaintSj teachera or vocational- 
instniotora ' 

■ ■ * ' ■ 


No action / 

.\mi) i . * 



t 



Note: (BSTS) -s 'Beyond Scope of Tte Stu^ 



HI 



\ . ' ■ . ! 



' GoniiBnte of Mewir 12 


Action teten 


Question 11 In ^rationil teis Mmnti "Does Ihe 31 iraluate wtete : 
; tfe prograni \m been Impleiiiintid is oM^nal^ jlmidf " Is thi sn as 
Question 16: "Doii the SE imluate wtetlur tha prog^ has teen aslrto mit 
' stated propm plsl" , if you can assume that the ori^ml plan InoMai ■ 

■~ ■ ^ ■ . \ . / , ^ • t .■, . , . ^ 


tetlonl5 
delsted 

i i 


Md to Quiitlon 11 in Operational ft^cess Initrupnt; .or m Inproveiiint 
over that wMch iei=|Iinnsd.'' ' "■ ' - 


Dilttidthe. ■ 
word "or^Mly" 
te lastion ll - 


Mcifcid the phmse "peopli Involyid in anfl/OT affected by" in Operational 
frocesi Mipoints was ^^''toQ iip^^ , ■ v . ' ; ' 


No action taken* 
; researclir" 
intendidtobe 
non-catiiorical ' 


Etelete :ftiistion 1? In ^rational fr^m MMint; 'lltoiSitla SM ivaluate 
'iithir til propani staff identlfli's ind rssOTds wMt actual^, takes place: 
during the dal^? . operation of the pro^?"'. Ms would require an inter^etlon , 
aMlyais, ' / > ^ 


-^'Quiitlml? . 
, . dalitad ' 


tiltioned how it would be poislbls to mswer Cpemtional Ppocess Quistion |8 


tetion' IB ^ 
.(telitid 

=^ — ■ ij- 



(taints of Mevir 12 (Continued) 



toitlon tin ilpiifleinei of Dpirattol ftwe is Quistlm 15 



Questionid what would bi in aeeiptabls Meator thit a teiolar M thi , 
mm^m qualifieations to work ^th m stiudinti (*iritiW ft'oceii 
Initoient Ration 112) , ' i \ 



toOTd tetlon Ik in Flaming Mm Mmnt\ "Deis thi Si etluati ' 
wfithir thi lM pp^ is iirving on^ those studinti Mble to iUGCiid 
in tliir' voeitlota-aduoation pw^^ 
desipid to niit the needs of M studen^^ 



mn Questian I? in PlBmlng teiss Mniinti "Does m SI e' 
whithsr tii urist rieds of indlidduil students influence the 
.respective progmn goili m objectives?" to dliarly identify ni, 



of thilr 
of"tlilr 



qperatiomlJy dste "PlaMni teesses" iid "C^rational tesses" 



M| queition(s) rekted to .assure the Mm «|ininti of H/D stuflints have 
been met in Placing md Operatimal ItoGesses istMBnts. * 





Action Tikin ; ' 


Nb cQflnnti * ^ ! ' ^ 


r N/A 
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PROPOSED USE OF THIS mmm 

This Instryiiient will be used for co^ a qualitative content anilysis of immints ussd : 
by state iducation agencies (SEAs) to4yijliati the' affictivinp of local idycatibn 
(LEA) yocational iducatiort^ograms for the handicapped and; disadvantaged itudint, ' ■ 

. ■ ' • . ' ' ' , ' , » ; ' ' 

'"' ■ • * , ' : • . . ' ^ ^ ' ' ■ ' 

»'.'■'-■■ . , , ' " , ■ ' 

The.actiiil content analysis of SEA docunients will be doni by JirFrasier. This instrument 
' will; not bi mailid to a SEA nor will a SEA be requested to conduEt a Mntent analysis of 
/evaluation documents using this instrumenti 




^ . : . ■ , ..■ . ■ " \ — ' — 

■ - r-- : ■ ■ ■ - 

^ ■ ' ■ i. : ' . • ■ ■ ■ , . . ' •:-'■ = 

' " ■ ■ " . . : ■ ■ _ 

, . iat types of evaluation strategies iri biing'Mid by SEAs to evaluate the PWIfiG PROCESSES . 
^ of LEA vocational ^education progranis for the haWcippid and dliadvantagid studenf? ' ■ 

^ ~ ' \ . . .. „ . . ' ' , ' i . . ' , 

■ ■ ' ' ■ ■ •' ■ ' '■'■I , ' ■'■ ■ ■ ■ ■ ' ■■. 

. : '■ . ' ■ : ^ ' ' ' : ' / ' . . ' 

"' ' ■ ■ ■ ■ ■ ' : ■ ■ ■ ' ' ■ ' ■ , ^ ' 

i:*^ ^. ' ■ ^ : . . . , ■■ ■ ■ ' ■ , .; w ' ■ . . 

■'■ , 7 - Prepared by' 

' - ^ ' ; ' ■ . James^ Frasier • ■, 

. ' , ' ■ " ' Agricultural Engineering Oldg. 

DipJ, of Voc Ed. nechnology 
; ■ : , L. • Unilerslty of yermont ^ : 

■r--' ■ r-'-'^-') -V - ;-■ - :- v-- - v - ■ - v ', ^^ -■ ■BurlingtonrVT ,::. 05405- 



1 havi diifilspid the ^uistloni liltid balow t§ lilantlfy wtieitiiP lES 
iVilyitiQn.diicyiiinti i^dnss ths lyalyitlon of prpgriiD OPEMTIOHAL PRQEEISES . 

1. crltieiily fivli« and CMt.Bn the appraprlititiisi of iigh 
qyiltlfl<i VilyiJfi1dintlfyln| prggrani OPEMTIOIIAL FMCEK^^ aiid, 

^ L offir additld/iil qyiitlsni yey think eoyld be pd to Idi/itifjf. 
. . pFogrioi OPEMimL PKISES, 



DD€i the SEA iviluiti uhetliif ihi prsirioi hii biin InipiiiPintid ar 
erigjnilly plinned? 

Doii the SEA ivaluate whether peppli invglved In ind/Qr iffictid bjf 
igrim are mh of the pfpgrifn^ pyrpoie(ij? 



O 

AO . 



3. OOil thi P iyilyati Hiiitliir piopli InvsWid jfi ind/or iffectad by 
thi pfogriffl m mti pf thi mnim provided by the program? 

i pois the IfA ivalyatiiNhito piopli Irivolvid InUnd/or affected by ^ 
the prggripi iri In igriifjunt wlth the ideqyacy of Initructiona] 
fiioyrcii iyallibli for uii by 

1, Joai thi SEA iyilyati;^tiithif thi prograin is haying in iffict on thi 
pafticipaiiti- pirftniianci In tfelf riipectlve voEational iduCitlQn 
pfggraifll?' 

S, Doii the SEA evaluite whither thi prograni has bun able to m$l 
-\ itited prdgram gqali? 

h Poei the SEA ivaluatg whether thi proyriDi stiff Idintlfies ht\i nmii 
what ictualiy takii plaei duflnglhe dally opifition of 'thi prograjii? 

B, D6is thi SEA ivaluati whither thi progrini itiff fUDnltOfi on a 
. .4. contliiuoyi bull thi potential of pirtkipants tQ luccied In their 
reipictlvi yqcatlonal progrinii i\^mt the prgiriin? 

.9, Doei thi SEA ivalyiti whether piopli Involvid In and/or affected by . 
thi prcgrafli bilievi thi instryctlDnal offiflngi are avallaWeHQ thou 
dliidvantapd and handicapped itudints ^t In mid of thi prograni? 



10, Does the SEA ivaluati whether po 



the program biljivi thi Instruct onil offefingi avillabji an 
ippropHatilyjatehid with ident ^ 



1i iDvolvid In and/or affected by 



12. 



fled itudint UfiiDit niidi? 



Ooes the SEA ivaluati whithir thi progfim hii an appropflati tiicher/ 
pypi] fitig to sirvlci thi ynniit needs of itydeht partlQipanti? 

Ooes the SEA ivalyati whithir the progfaifl-l initryEtorlsl has the , 
niciiiary tiachir qyallfkatta to work lyccissfulW with handle 
diiidvantipd vQcationalitydfntf' ynfnet^fieeds?-^ 
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The laiiiB «|f lii.nMthen*' fomit uiid in thi plinrilng Instfunifnt will be 
developed to Idifitlfy the typii of ivilyitlQn UPstf^lii bilng uiad by SlAi 
to evaluati pfogfiiii flFEWTlONAL PMCESSES. . 



Ai yoy riviiw the qyeitions on thi opposite ilde of this pagii w|]] yoy 
pl^si^ ^ ■ , . ^ ' ^ 

Ij Subitltyti quhtiflni within thi Initryment ft^^ 

I] critkilly review and coRfint on thi appropriateness of yiing 
the "If ye5,.,.,tlien«fmt tii Identify 

3) if appfDpriati, offif an altipnatlvi fonPitJor yse In identifying 
types of iviluitlpn itrategieruilng SEA ivalyatlon docyiiiintSi 
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: 1. tePS thi m evglysti whetjif LfA progriHi pali and sbjiEtlves havt bein 
j Inpliffiented as planfied? • v v 



JF-YK^ yhai SEA evalyitlgn itratggy !m\ i are used to i^imm \Mkf 
/: Ilk mim pall ifld ebjictivgl hivi bien iiiipliiiiinted is plannid? 



jEjiSelf-eyilyatisn- 
Umm Eviluatlen Tiiiii 
Jxtiriiai Eviiuatioii by an ■ ixport • 

Jndepindint Third Party Eya'lyatiflfi 
Jvalliation by Btati teultant 



liling IH divelapid initfUHnU 

^jslng SEA dgyglapid liiitfyiii^s 

_ jsjng nelthir M Qf LEA developed 
. initrui!i.efiti ■ 



D n mi in evaliiatiQn Bf diiadyintigad prsgrii 
11:=! mi in ivaluatisfi of hafidicappad pt^nm ■ 
D/lj^l' used in ovaiuatlsn of both prQirais 



inKi iatiyalyition strategies m mi tojgjlfytoj LEfl pfBgrin) goali 
and ebjictivii havt bien IniplMted i| plinnid? 



OliadyintiO^ li andkappod 



yfitten Slimy of - itydanti 
" ' " " " v - parints^ ^ 
. -adininlstritsrs 
-vocatlDnal faeylty 
-pypil luppoft perionnii 
^prcigraiii idvisory coDtttii 
^Dthe r__ _ 
?mm] Interviews with -stydtjits ~ 



Mm of itydgnt 



Mm of US 



-adiifiiitratQfi , 
-vDcitlsnal fieylty 
-pypn support perionnil 
-prsgrk idviiory conittii 
^stiii r 
-ficpr^l 
=tiit icons 
^drap-DUt ratifil 

^nd 5 yeof plans 



^Orifit tolicitjoMKFP) 
„ 'M 'of year pregraiii report 

^Dthgr 

Wcw ofiES^Censyltant'l viiip ' 
=CDiiiijliajice reports 
^Hiniiiip prQgriin slOL 









::li-^iOlitLEM- 


ioies|«0"-S"ligt prpsaiit u^ 


llHOlit 



1 



jng: iat evaluation itratigiqs m yiid M verify ik gffsetiviniS!. of " 
LEA pfQgraiii liiiplirentatlsn of plinngd'pali and Bbjictivii? 



', \ HHttenjyryoy of -iMei)^^ 

, / /pirgntl . 

^ ^ " , ^iWniitf^atdrs 

-vpeatignal facnlty 
'pupil suppDrt perionfiel 
■ * -program adyisQi^ mittoe = 

' "^sthc f ^ . 

Piriflnii lntorvieHiwith*stydents "^ - 
-parinti^ 

. : ^ -adiiiiniitratdrs \ 

. 'VQcatigiii] faculty^ 
, -pupil support pirionni! 
, -prQiriinldvliory cdiiiiiittee 
' "Pthir 
tevliw of itydint -rocarHl" ~~ ~ 

-dfop-pyt ratios 

'Qthar \ ■ 

Mm of LEA -1 arid 5 year plaiji " 
-Grant /lppli^tion*(lFP) 
-End of yearprsgrani riport 
-Qlhef 

Mm of m ^miiMfuMtm^ 

- ^taipliancoriporti ■ - 

4lin1ra prqgriiiiitafidirdi 
-other 



plsadyantagid {Wjif jed: 



1 n ymEi 



fla tepoflse Not present in Ootyiflt 
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INSTRUMENT A: PLANNING PROCESSES^ 



' . PROPOSEP USE OF THIS INSTRUHENT 

This instrunient will be usid for conducting a gual Itati ve content analysis of docunients used 
by state education agencies (SEAs) ^to evaluati the effectiveness of local education agency 
(LEA) vocational iducation programs for the handicappid and disadvantaged student, 



The actual content analysis of SEA docunients wi|l be done by Jim Frasiir. This instrunient 
win not be mailed to a SEA nor will a SEA; be requested to conduct a content analysis of 
evaluation documints using this Instrument. 



^ MAJOR QUESTION OF THIS INSTRUMENT 

What types of evaluation strategies are bilng used by 5EAs to' evaluate OPERATIONAL PROCESSES 
of LEA vocational iducation programs for the handicapped and disadvantaged; student? , 



Prepared by 
James Frasier ' 

Agricultural Engineering Bunding 
Dept, of Voc. Ed! arfd Technology 
University of Vermont 
Burlington,^ VT 05405 



. thi qyiilloiis ]\m Mm to Idintify mm lEA ■ 
.iviiuitifin docuDinti iddnis tta ivalyitisn ef prggnij PLANING ProSK , 

te jffiy riid iliesi guiitiBflli will )ou pliasi 

). efltleally m\&^ and Mnt an tlii ipproprlitinisi of m\t 
— quiltlQn'i valul jn idintifylng prsgfiiii Kmim PMEESSES; and, 

• " Qffar iddltional pstioni yoy think ccyld be uied to Idintify 
prep^ EIANNINS PMIKIL 



1. Dqii thi m eyalyati tttiithir thi LEA kimU tfMltn Idintify locally 
iiatoti^id rtioypeii ind/DF lirvkil? 



it Dsai thi SEA fvalyati iffDfts uiadr^ thi LEA tD ulilizi 
handlcappid/dlsad^^antigid fisQurcii and/dr sirvjcii? 

h Does thi ilA evilyati iffartsiadi by the LEA proirim to idintify itydintl 
neidingjhis pfQgfaffl.Mn Ordif to mmiM thiif' vocitionil iduEitjon 



, 4. Oqii thi SEA ivalyati Mlkt thi LEA pragriin 1| sefylni thou 

handicippd/diiadvinUgid stydints ynible to immi in tilr 
^ ; '^ocatiflnal ^ 

fy 5, Dsn thi SEA ivaliiati Mkt tha LEA prdgrini issasill the unmet nieds of 
iich Individual prograni pirtklpant pfisr Es providing lervkis? 

6* Does the SIA awlyati M^tr indlviduaji tnowledgiibli ibput thi itudent 
' ' (i.g, pifgntii tireheri/ptiil idyeatorsj are involvid In ilieiilnint 
: ^ aetivitles thaNeek to Idintify th| itudint-| ymt niids prior to ttii 
prograiii providing servlcii! . ^ ' 



; h Dots thi SEA ivalyate whethar thi ynmet niids of indlvldyil itudenti 
Influince the fonnitlon of thiif riipeEtlvi prografli goils. and objictii/ii? 

8. Don the SEA iviluiti whethir Ehi unlit needi of individual liydenti 
Influinei the fsnition of thi LEA prograniU ggili ind^objigtivii? 

9. m the SEA iyilyatuhither thi itydint'i VQCitionil initrydtor is 
Invglyed in thi plinnfng which divilppi goals and ohjecihes for meeting 
thi student'i ufifflit neids? 

10. Ddii the SEA ivaluiti^t^hethir individyili tolidgiibli about the needi of 
"'■ hindigipped/dliidvinti|id itudenti (hindicappid/diiadtfintaied worbri, 

advisory eonittii mimbirsj an Inyolved in thi plinning which infjuincii 
. the fsriatian of the LEA progriifi'i goaU and objictlves? 
. . ' ■ ' i ■ ... 
ii Boei the SEA is/aluite thither tht LEA has established tjiilinii for 

coiiipliting major progniii goali and objirtivii? 



12, Doii thi SEA iyelyiti jhithif thi LEA hii I plan fop^pliiiiinting thi prograrn'r 
pll and objictivis? 



1 havi diyilopid an 'If yei,.,, jhen^^nsi™nt fonnit to idintify thi 
typii of lyiluatfon stratiglei being mi by SEAi to iviluati prQqfain 
PLANfilNO PROCIISIS, 



Ai ysy fiyiiw thi esntint analyiii InstryinentVfOTati will yoy pliasr 

ipriatiniis of 



1, critically review and cofimint gn the 
the If yei.....thin' formati andp 



I if ippropriiti, pffip in llternativi foniiat for. uii in idlntlfying 
typii of iVilyation Stfategiii uilng SEA iviluation downts. 



mENTi AW iyodESTiofiS! 



15 



h tell Ilia HMviluite whil^^ the LEU hai inide effofls te Idintlf/ 
locally m\mk Mkippgd/disadvantigid rmtm ind/dr iirykgi? 



m 



NO 



gM\ M lEA ivalyitjon ilrili^ f^rgh are yiid to dilimijiji ^u^t 
thi LEA hii nidi tffgrtl ft f3if!?y IgEslly m\Wi mmm 
and/or sirvlciil 



JIA lilNvilyitiofl . : 
Jiiterna] Evalyaiion ]m 
Jxtemal Evalyatlon 6y an "giipeft' 

Jndfpindent Mri Pirt^ Evaluitlsn 
Jvilyatlon by State Consultant 
Other 



using LEA devglopid initfuiBinti 

^ Jlng SEA im\mi Initwnts 

Jailing nitther llh gr LEA divelgpgd 
■ jnstfuifientl^ ; % 



= D H mi In eyoluatidn sf diiidvantiiad pfowl " 

11 H mi in lyalyatjon of handjypped pfpgranii 
W. n mii If) ivalyitlon §f hath mm 



Mhat ivalyatign Itfateplfs an ysid tgvirj^how the LEA his 



luEh locally avillahli Feioyrcii and/flp sin 



Diiadvintiqed Handkappid 



, , yfiUinjyrve/ Qf vityte . 

- pariiiti 
^idiiilnlstratorj 
^vocational facyi^ 
-pupHiyppgrt pirsQnnel 
'prsgrani idHigry connlttia 
^otlii r 

. Pirional Intirvfiwi nlth *itydints ' 
-parants f 
-adinlniitritdri ' 
-voeatiohii ficulty ' 
" -pyplliypp^tp^fssfinil 
-prDgrainadyiiQry mjttei 
-othir « 



Rfview of stydint 



Bevies of IE) 



-ricordi 
4eit learei 
-drop-Dut ratloi 

^1 and J yair plans 
4rint Api^lkitlon fPj 
4nd qf pr prdgrai!) fepgrt 

SeviiH gf SEA -CDniuTtintTMFHpERF 
^Coiiipliafice reporti 
■^HinliiyiH pfggr^ standardi 
' ^Other 



? H UNCLEAR 



lis kmm H Not preiint in dQcyiiient 
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mi i/hat i^aiyatigR itratiglii art yied te vfrlf^ thi'effectivineii of 
LEA prmium tD identify lyEh jocllly aviiligfe riiQiircii ind/or 



Uriitgn lyrviy pf -itudentl 
-pariiiti ' 
-adiiiinlilritors 
^ *voE3tional ficulty 

'pypll iuppdrt ^mmm\ 
■ ^prppiiidviiory cpiiiiilttee 

ffienil interviei^s Hitfj :-ltydgnts " " ' 
-pafinti 
. -adinlniitratQrs 
' ^ ■ -vocatiDnal facylty 

'pypll lyppdft pirsdnnil 
-prepii adyiiQry cMniittti 

fiiviitf of itydint -ricfirdr 

*drop-oyt fitios 
J Esther " j 

Seview of LEA -I and S . year plins ^ 
.^Grant AppHEitign (P) 
-End of yiif progfii!) repQPt 

-OihQf _^ 

■ Mm Qf SEA ^onsylMPTWlt ripsrti 

'•tapliifirtripoft^ 

-NiiilM pfopiii itihdafdi 
^gther _ 



Oiiadyintiggd 



'Pid 




H mm 



1(0 Mfmi H m presiilt in Docynient 



EOmENTI EOfjCEilNO COmi AMirilli 



m 



APPm©IX E) 

im CONTEOT AMLYSIS IKB'IEtMJT 
FOR 

mMJUATZOn OP PLANNTOQ PROffiSSES 
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. ihruiiiHwdiifmnaTr^ifltijirm 



^^hterfiil |yiiyatl§ii till 

' ji ■ . 

__faEinial Ivilyitlfiii Iran ^iipirt* . 

^^IndfpffiElfflt TMrd Party tuMka 
__ ■ Evjlyjtlflfl fey Stite Cpnsyltliit ' 



Jiing UA dlVilepid initwRU; 

jitng SEA dfvelpd IflStnfflenti 

js^9 niltliiF m or Ltt imh0_ 
inltFiiintI = 



P *i uHd jfl ivaiyitlpn of disadvintagid prDgrmji 
H n ysid in m]m\m of handkipped pf^gm 
/Hj[yiid in evilyatjpn of bpth prpgrafni 



soygOt tQ Idgntlfj lueh laealljf ava|Tl#fei| md/er sir^iiesf 



Diiadyanta^ld Handleappid 



HrlttcMyrvijf of Mtudintl 

MdnilnistrittPl * 
^ •vacationaj ficy]^ 

^ 'PVi^idyJigryceiiltlft 
Piflflnil Intirvliwi Njt^ -itydintl 

"idniiniiti'iton 
*YOcatjpnai faeylty 
*pypi! lypport ^mmnt] 
^prpgriiidylsory epilitll 

teviiw ef itudiflt ^recpHl 

-drgp'Oyt ratloi 

'Pttll F _j 

Btviaf of LH-lanf^ yiar plans 
; ' ^OflnUppiititipR (P) 

'End of fiir prigrai^ riparl 

: Riviiw of 3H ^Cpniyltint'i ylsll fipprts 
*tep1iin^i rfpprts 
'fiinta pr§irif!i standirds . 




fojBpnse ^1 Not preiint 1ft dp^yi^^nt 



^ iH||i Kdlt lyaluitlan itritiiiei are yi?d tgjerlfy tk ^fkaUm i of 
lis pTOdyrfs tP idintify lueh IscaTly mmU kmmMht 



of ^ 



lenficfi? 

HHttpR iynijf §f ^itudenti 

|^tei|tf|{fir| 
HocatiDnil facylty 
'pypll syppoft pifionnei 
'Pragrp idyliQf/ cMluee 
^Qthif ^ 1> _ _ 
Pirisnal Intii^ltwi i^^^h^itoldifiEl^ ] \ " 

'adilnlitra'tpfs 
^vocatlgnil ficyly 
■ *pypil lyppQrt pirsonnil 
* #Qpi idvliSfjf cqiiittii 

Riyfiw of itydinl -racardi — 
•teit scorn 
-drgp'oyt ntlgi 

teiflK of Ltt «1 and b yeir plini 

» lOrant ApplicitioR (RFPj 
^ifid af year pragf as repoi't 

tevia* of Stt ^teyltarii's viiit nporti 
'taplianci ffporU 
. 'HiniMpfograif lUndardi 



Dlijdyantaatd jiindigjppej 



1*1 m 



No kipmi H Mi present in Do«n{ 
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I. Dafi thrifA iVilyata iffof ti fflade by ilii LIA to ytllfii locally 



jnij f What SES gyjiyatien Ufitf iy ftfnBti are uitd to Mm\m w^etfur 



LIA SalNviioitlQn ^ 
mm] Evilyatlan Tian 



_^uilnp LEA diVilopid Instfyiiiiny . 
. _ y'iing SEA.devlloptd initrunients 
1 faternal Mluatlofi by an "ixpiFt* ^ yilng^T|fiti#MEA jr LEA developid 

Indipindfnt Third larty Eiaiyation^^ _ ot(^ 

■ ^^tvaluatipn by Itati Cqnsultant, ^ . 



: 0 «i uiid in fvalyatfan of diiadyantaged prdgw 

H n yied in evalyatfofi Df liindjeappeOrogfilii 
D/ii =1 uied jn ivaluitlgn of both mmi 



lyalyatien strategies ira yied to veri^ hgw tha LEA ttai 



\5 nif^ ifforti ta utUiii luch nloyFces and/or sarvkes? 

Disadvantagid Hindicipped 



lyrvey of ^Jtudentl 
T ' ^i;: . ' pifents 

^adi|1ntstfatQf| 

yoeat; 

H ■ ^^pupil iuppsrt personnil 

^ - : .'!:^-prggrii addiory .mlttii^ 

' ' \\ = = t «pargnts ' . ; 
,^ -aWnistrators 
' ^vocational ta|ty 

-pupil iupport pifsonnei .:. " 
I ^prograiii adviiory conlttei 
'^gtiii r ■ ■• • ^ 
Revlai Qf itydint -nmk 
-tilt scares 
fdrop-oyt Fatios 
-Qtlie L ^ -^^.l. 
rRtvlfw of LEA -1 ind iyeir plins 
■.■■r:-- . -Grant Sppiication^f^^^^ 
"^x- \ .t -Eiid,Qf yiar prdirani fiport 
■ .. " ' :-/''Wtiir^^_;^^^^^_^ 
^ ijRevliw of lEA'-Cpnsyltant'i vtut repofts 
■0 ' ■ " i-£oiiipl,Unee reports ' 
' ' . ■ ^ ^Hintan) pfapm itindN 



I 



i n KES 



fio' Riipofisi ^1 Not priiant In dDMt 



■ lF_]fE| ; Hhat ivaluation stritiilis are usid to virlfy the iffectlMtss Qf^ 
LEA prD§edurii for ytjilzlnj lu^h riioyriei and/or iirvtets? 

Hiadiiahtigid Handlcappid 



lyfvay of -ityderiti ] 

\ ,=adSi1n1itraigri '"/^ 

-vocational fawltyy • " ■ 
, *pil suppDrt piiipnflil ; ■ 

'Pri3iraf!i advisory M^^^ 
'Dtiif r_ ' 
■PiPional Intirviiwi with -itydinti 7 

-parifiti ^v. 
-idialniitfitori 
^vocitloDal facul^ . 
' ' -pupil lupport perionnel 
-propi adviiory cort 
/^Qthe r ■ :■ 

Riview of studint -riepnds 

>ttit %mm , f ■ 
=drap-oyt ratloi 

RiVieH of LIA -1 and i ygar plans 
■ ; -Grant Applleation (RFF) 
>End of year ppograii riport 
. Mother - ' . ■■■ 

., Rivlftf gf iEA -ConsyhiRt'l visit ripofts 
■-CoiiipHanci flports ' 
Mm pfografii standafdi , 
. ; -ottiir 



Riiponit ^1 Not: priiint In teiient 



rBCQHMlffi MEliTANjiL^SIS; 



I 

4 ' 
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0 



.3i Ml thi SEA evilyiti iffortl mk b| thi LIA to Idintlfy ttisse itudents 
^ in mi of.ipiElil lirvkes or wwmi In ordir to TOed 1n ttelf 
vkitlmKetotlon pfsgfae ........ 



NO 



Itudints niidiny the propilal^TJentlflid? 



jtt iilf*gvaiuatlon 
Jxtirndl lyiluition Tiifii 
JxtiHiil IvilyitiBn an ■iprt^' 

Jndipindifit Third Party {yajfuatlon 
Jvilyation ^ toniyitint 
fithir 



Jling Ilk diyiloped IhStruiiienti 

JlinflEA diviioped Ifiitryiiiinti 

jlinf hiittiir SEA or LIA diVilopii 
Initfiiints 

Dthir ^ 



0 "s yied In lyaLyition of diiidyantagid pfograRis 
H n uiad in ivalyitloHf hindkappid pfograffii 
i/H ^1 yiid in eviluitfQn*Qf both prc^rais / 



IFJSi What ivalyatlDn itratigiei ari yied tgjiri^ tow the LEA ditinnifiis 
tiiat students neidlng tlii program ira Idintified? 



Diiadyantagii Handieappid 



tiflttin survey of * itydinii^ 
- parents 



M 
N 

OS 



-idffjinlstratQrs 

-pupil syppDrt perionnel 
. ^progriiii adyiiory coisiiittii . 
-Ottier . ■ 



FeriQnil- Interviiws Nlth ^slydinti 
^parenti 



Mm of Itydint 



-administritors - 
-voeatiQnil faoylty 
^pypjl lypport pinonnil 
-prQgrain adyiiory cenittBe 

-Qther ^__ : 

-^raQQfds 



-test scores 
: - ■ ^drop*6yt fitifii 
-othi r ^ 

' tevlew of LIA -1 and 5 yiar plans 

-drint ^plicatjon (Pj 
-End of year progfafli report. 
• ■^othe r _ ■ _ 

RivjeH of SEA -Consyltant'i vlilt reports 
^CBfflpHinci repofti 
-Hlnlpiym prograin standafds 
mother 



K ^s yES 



. preient 1n datant 



V YB i tijiat ivalyition.itratigles are yiid to virj^ the effigtiviijeii of 
: LiA pppQidyrii for itatlfyifig itydenti nedtni the progr^iiif 



i/Plttin iyrvey of •itydintr 
^parints 

' ; *'adii)iniitratori 

-yocationarfiiiilty \ 
'pupil lyppoft pinonnil 
— -prograniidvtigfy gpfiiiitee 
■ ^Ptto f 

Pirional Interviews >ijth ^itydinti " . " 

^parents 
-adiinislraton' 
. "Vocational faculty . 
>pypi1 sypport pefsosnil 
-prograi idvlspry conittei 
'Otiii r 

Raviiw of student -record! ■ ~" 
=teit icoras V , 1 
*drop-out ratioi ' 
''ptfei r ^' 
Jeview of LEA 4 and b year plans 
* -Oraiit Appiication fBFP) 
-End of yiar progn report 
-otiier 

Riviiif of SIA 'Consultant vlilt raporrfs 
-Coiplianci riporti ■ 
4\n\m prpgriin standardi 
-otfier_ 

\ .: . — 



Oilidyantagid Handicappid 



ii^s TEl 



' ' VH MZim ' ' Npliiponsi'^i Not piresent in Qociint 



MNTjjMii mm mm\ 




i Dsn thi SIA ivaluate whittiif ilii LIA pgfasi 1i iirvliig ^ thou 
Wlcpid/dlsidvintip^ 

VQoatisfial education prograin? . / ' : 



Ifll 



fig 



IL^l tfhit SM iifilyfltlQi) itritg3j/ fdnnaU irt ysed tg tfatinnfflg whitliir 
' cnlj lycli itudenis iri being sirvid? 



JEA lalf-iviluitlan 
Jxtefnal Evaluatlofi Tsiffi. 
Jxtpil Evalyatiin by an H^trV- 

Jndipindint Third party Evalyation 
IvilyitlOR by Statg taltant 



_ui1ri| LEA diVflaptd Irjstrwnti 
_ij|ini lEA divilopid jnitrininti 
^usini neithir ilk or LH diVglepii 

oth^r 



D n ysid in lyalyitidn of dliadvantaged pfoirims 
H n mi in ayaliiitlpn of hindlcappd prayraiiii 
D/H used in iyalyitlsn of both prograiiii 



IfJif M mMim ifratigies iri ysid tPVirif^jg tk lih ditimlnir 
. that only lych itydintl ari fifing 1^^^^^ 
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Jrtirna] l^iliiatlon 1m \i ; 
Jxtarnal fvilyat ion by in ■iKpirt" 

Jndipindint Wi Party Eyilyition, 
jviluitlan by !tata CDnsultant. ., 
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using m\\kf E|A sr LEI divilgpid 

■ initryinints ' 

^ ^ Other 



D usid inivlluition of disadvantaged prQirliiis 
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-Grant Appjbtion(P) : 
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__^E)itiFnal Ivaiyitlo/i Teaiii ■'■ 
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_ . ' *othaf 

Mm of SIA -CQniuItant's visit reports 
-toplianca raports 
-HinifnyiB propin itandirds 
-stfiir 



i^HES' iH'mim k teppnsi m pnmt in taint 



mim mmm mm mmm^ 



ERIC 



8; 



186 



NO 



ptepie, nvelned in anpriTficiriy tin progriiii in awira of thi 
proscp'i purposf(ij? . . , 



;_ L!A SelNgvalyitlM . ■ ■' 
^^EitterhallviluitldnTiw' / ^ 
. ^xtifiiil EviluitjDn hy -in *eprt' 

. Infep^ndint ThifOgFtnyaluatiQii 
__JvalyaH6ni)y Matt CDniyiynt 
Dther • ■;■ ■ ^ 



^ysinp LEA devildpid ini W 

Jilrii IM diVilopaf ini^rt 

%\f\^ niithar llh fif.LEA divelopad 
Imtrynienti ■ : 



;liir 



P^s mi In evDlyatloR of dlirtvaittipd prQprii 
H *s uiod In avaluitisR of handtcappDd prDgrinis 
D/H n mi In iyajyatlgi) of both programi ' 



ILl' ilhat evijuation itrstigits ari uied to verifyloN jiQDle InvphdHn . / 
ind/QF affected by the program m^mri of thrprggraBi'i pyrpDii(l)? 



Diiadvantagid '■. Handleapi 



WUen syry^ of - itydents , ^ 
irints . ! ■ 
fiTOitritsfi 

*pypii luppoft pBriannel 
. ^prograiii adyisofy cQntttei 
-QthdrJ 

Personal Intifvii^i with -iMm 
-parents 

/ . ' ^adminiitfiton . * 

. \ -vocational faculty^ 
/ .iupil lyppDrt pifsofinei 
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-prograni advisory copittH 
■ 'Otlie r , 
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^drop-out ratloi 
' -Qther 

toliw of LEA and 5 yiar p1anT 

*af\t Appllcatian [P| 
t -End of yiar prograii) fepijrE 
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' . ^ ^drop-out ratiol 

-Ottiir 

Reviiw of LEA -1 ind ^ y^ar"pians 

-Grint AppHcatiDn (RFPj 
Ajii of year prograip riport 

-other 

Review of EEA -Cdniultant's visit"rprti 
-tapliance rlports 
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adeqyacy af .iflitFyitlonal raieyr^ii avliJiHi fsf yli - ' ' 

by Uie pirtidpiiitsh; ^ ^; , . 



jQii Ihi ilA iyilyati MhfEher the pU iiid objieljyei 
ptiM tg hilp thf itydintiyg^^^^ ' 
progf^ hm btin l^lmtgd By't^ifrggrii ai planned!, 



I* \SQerthf jMeifilysti Mh^^ ImmUf 
. Ilil piflkipated Id ^ the lipjiigntltidn of prograi gglil I 
pbji€t|vei plaMid t^hilp.th^ in i ' ~ 

vdea{{|nii|fpgrai? 



h k^^^Uipihrn MhithiP the prsgr^ li Mnj an 
ifftctWUipartUjpani'rpi^^^ Inhli/hef 
ypeitisffii idysatlOi) prQgfii? . ^ 



ten the IM evllyite iheihif^pliple Mvid In arid/df 
affected b]f the p^gras ^| levi the Inilryillenal 
sfferlngi ivillable in appriprlitlly iaUliid ulth 
' lstydinlyns|et|jieds? 



H 



^H the S!A;lyl1yate vhether t^e progrii hii an 
^preprUtg tiiciief/pypn rails! 



19. D§ei Hi m\uii whether the prggraa'i ffislnjcisFlsj 
hii the ti|£hir ^yillfleatlgns Id Hort Nlth handUa^ped/ 
disadyantigid siydenisl 



2i0 



■V 



_ ^ mn 

STAraS V^mOOT E/D W/AWATIOn SYSTH^ 




1^3 



Your state does not have a special needs aval uati on system : RIGHT? 



bid you state iver have a speciaT needs eval uat ion sys temt " 



Are special needs pro Jects and/or services eval uated withfn 
or as part of a larger aval ua t i on sys tem J n your state 1 1 ke' 
an evaluation system for^aTl -vocational progrims every 5 years? 



,5 



Are you planning to develop a * ipeci al . needs aval uatlon system 
or are you in the process of developing one? 



IF YES: Who are you worki ng?w1 th to develop the evaluation system^ 

i 

- ■ ■ -■ - - - - - ■ - 



IF VES; Can you give me what you consider to be the purposes 

for eval uati ng .special need? pro Jects i n your state? 

. . . ■ T . 

^program improvement 
mee t 1 ng Federal evaluation mandates 
' Improve program pi anni ng 

__determine. fjjture funding 



IF YES: WILL 1t be part of a larger state ev.aluation system 
or separate from it? 



, FOR 
STAES Willi M/D mPVJATLm SYSimm 



J 



-1, , Can -you please^ tell me what are the purposes for eval^tlng^speclil . fieeds projects 
; , and services jn your state? ^ ' ^ • / . / - 



-r, - ^ - ^ / i mprqvi program^lartnlng 
' ' -^ ^ ' / ^ ■ d etarniine future funding 



^rogranr tmprbvemint . " ^ : 

i"^ng. Fideral evaluatian .niandates- 



2, Do you think that your stati systeni^ for evaluating special needs pfojects/and ^ssfvicts 
is able to determine^ the effecti^veness of local project pi a^nhi ng processes ? ' 



< 



On a scale of 1 to 5 — s'biing the hlghtsf -t— what number value would yOu give 
.to your state system's ability to detirniini the frffefftivaness of tDcal projeqt . 
*, pi anni/ig processes? . " ' .v^, » . 



5 
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ERIC 



3. Do you think thit ^ur statr iystem for evaluating spicial niids proiacts and ^ ^ ^ 
ssry:ica,s. is able to datenriine the effectiviness of local project optrational procaisis? 

.(Prbbet /Can you give 'me some examples?)." ii^fe. ' ^ 



a scale of 1 to 5 S being tht hlghist^ — what number value would you 

give to your state system ' s abi 1 i ty to determine the effecti veniss of 1 acal prpje et ^ 
Qperatlonal processes ? ' ^ , 



s 

.1 



4, Were ther^ any difficulties that you on^your state ecountared In developing 
your special needs evaluation system? ^ _ h ■ - ■■ ) 



5. Have you encountered any difficulties in Implementing your special needs evaluation 
■ system? " - ' ^ " . ^ * 



21o 

1^17 



ERIC 



How do you as 1 stitt consultant use the ftsults of specml needs evaTuatlons? 



How do you as a itata consultant think the local level vocational educators use 
the results of special needs evaluations? ^ 



Based on your knowlidgt as a state consultants "What, has been the reaction of 
locaV level personnel to your state evaluation system for special needs projects 
and sfrvices? 



In reviewing youn evaluatiofi document^ my Judgement is that your special needs 
evaluation is is not integrated into your state's 5 year evaluation of 
^all vocational programs , Is that correct? 



10. you pTi.ast givt me somt exainplis of the typts of 

•special netds projacts you are eval uat1 ng -with the 
Instrumtnts you stnt me? . • ■ i 



WELL, DO YO~\J ItAVE. ANYTHING YOU WOULa LIKE^TO ADD OR TALK ABOUT 
THAT WAS NOT INClyDED IN THE QU ESTI ONS . I H AV E AS KE D YOU ? . - T 



\ 



WELL, THANK YOU VERY MUCH 



OH, B'Y" HE WAY, 'BOB WATSON SAID HE WILL BE SENDI NG YOU A COP Y 
OF THE EVALUATION SYSTEM WE DESIGNED. IT HAS JUST 'GONE TO' THE 
PRINTERS SO I .WOULD THINK YOU WILL HAVE ONE IN AUGUST SOMETIME 



